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ABSTRACT 

JOBSTART, a federally funded demonstration program, 
offered basic education, occupational skills training, support 
services, and job placement assistance to young school dropouts. A 
study of the impacts of JOBSTART was designed to separate effects of 
the program from events attributable to other factors. A total of 
2,312 people eligible for JOBSTART were randomly assigned: 1,163 to 
the experimental group and 1,149 to the control group. Thirteen sites 
conducted random assignments over varying periods of time from August 
1985 to November 1987. Follow-up surveys at 12 and 24 months after 
assignment gathered data on outcomes such as participation in 
education and training programs, educational attainment, employment, 
earnings, and use of public benefit programs. An examination of 
JOBSTART services and participation at the 13 sites found great 
diversity within the general framework of the guidelines; longer and 
more substantial participation than that in other programs for young 
school dropouts; and similar participation hours among different 
groups and types of sites. JOBSTART had strong impacts on educational 
attainment — completion of high school or receipt of a General 
Educational Development certificate. The impacts on employment and 
earnings were encouraging for mothers; small, but slightly positive, 
for other women; and generally negative for men. JOBSTART was 
sometimes effective and sometimes ineffective in yielding second-year 
labor market gains in brokered programs and in-house programs; it 
also showed varying success in both concurrent programs and 
sequential programs. (An executive summary, 64 tables, and 10 figures 
are provided. Appendixes include additional data tables and 55 
references. ) (YLB) 
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PREFACE 



One of the major challenges on the domestic policy agenda is to improve the skills levels 
and em))loyment prospects of disadvantaged young people. For those without a post-secondary 
education, the I'^bor market situation continues its downward spiral. For youths who have 
dropped out of high school, the problem is particularly acute. While many ideas have been 
put forth for alleviating this problem - most recently in a rash of reports on school reform and 
labor market trends - there is no compelling evidence about what works. 

The underlying hypothesis of the JOBSTART Demonstration was that a comprehensive 
program incorporating many of the key features of the wideiy respected, primarily residential 
Job Corps - basic education, occupational skills training, support services, and job placement 
assistance - could succeed for high school dropouts in nonresidential settings. Success was to 
be defined as increases in educational attainment, employment, and earnings. Thirteen 
organizations were selected to implement JOBSTART and to cooperate with the multifaceied 
evaluation that was central to the project. Since special demonstration funding was not 
available, the bulk of program operating support came from the nation's employment and 
training system authorized by the Job Training Partnership Act (JTPA). Thus, the JOBSTART 
model received a "real-world" test by operating within the existing mainstream delivery system. 
MDRC - a private, nonprofit corporation experienced in designing, overseeing, and evaluating 
innovative programs - conceived the demonstration and had overall responsibility for managing 
it and for conducting the evaluation. 

This report, the third of four on the JOBSTART Demonstration, summarizes the key 
implementation lessons, including recruitment and pailicipation patterns, and features the 
impact findings from a two-year follow-up survey. In this survey, the educational and labor 
market experiences of youths who were referred to JOBSTART were compared to those of 
a randomly selected control group. Overa'^, the results were mixed. The sites succeeded in 
implementing JOBSTART, and JOBSTART substantially increased educational attainment 
levels, as measured by receipt of a GED (high school equivalency certificate). However, the 
educational investment has not yel translated into employment and earnings gains, although the 
results are somewhat more encouraging for young women than for young men. 

The repon also provides insight into different ways of structuring education and training 
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services - either sequentially or concurrently - but there were no clear patterns of impact 
differences based on how sites organized these services. 

A Gnal report, scheduled for 1993, will include the results of a four-year foUow-up survey 
and thus clarify whether, over this longer period, the educational investment led to greater 
labor market success^ 

MDRC and the policy community at large owe a special debt of gratitude to the 
consortium of 12 demonstration funders, the program staff at the 13 participating sites, and the 
sites' state and local funding agents. They shared a commitment both to enhance the 
ismployment prospects of disadvantaged young people and to build a knowledge base so that 
future policies can be guided by facts and evidence rather than speculation. Learning what 
works in the social policy arena hinges on the formation of collaboratives such as this, and it 
is our hope that the structure of the JOBSTART Demonstration becomer, a model for future 
endeavors aimed at alleviating poverty and increasing the self-sufficiency of disadvantaged 
populations. 

Robert J. Ivry 
Senior Vice President 



-vi- 



EXECUTIVE SUMMARY 



Although most young people move fairly smoothly from adolescence to employment and 
self-sufficiency, high school dropouts with poor skills are increasingly unable to make this 
transition. The statistics are stark: In 1990, only about one-half of all 16- to 24-year-olds who 
had not completed high school and were not enrolled in some type of education program were 
working. The figures for blacks are even more discouraging, showing less than one-third 
working. 

The consequences of these employment problems reach well beyond the lives of the 
young people themselves. Employers and the business community as a whole suffer because 
job applicants lack the basic skills needed to perform productively. Furthermore, demographic 
trends suggest a possible worsening of this problem: Over the next decade, the number of 
young entry-level workers will remain basically stable while the economy continues to grow, and 
an increasing proportion of these young people will come from groups with historically higher- 
than-average school dropout rates and basic skills weaknesses. From the perspective of 
government budgets, these nonworking young people are using services but not paying taxes. 
Government and nonprofit agencies must contend with long-term welfare dependence, crime, 
and drug abuse, which are tied to lack of employment success in many direct and indirect ways. 

The JOBSTART Demonstration addressed these issues by testing a program of basic 
education, occupational skills training, support services, and job placement assistance for young 
school dropouts who read below the eighth-grade level. The demonstration, which was 
developed and is being evaluated by the Manpower Demonstration Research Corporation 
(MDRC), was implemented between 1985 and 1988 in 13 diverse sites. Operating funds were 
provided primarily under the Job Training Partnership Act (JTPA), the nation's principal 
employment and training program for economically disadvantaged people. 

Tliis report, the third of four on the JOBSTART Demonstration, summarizes findings 
on the program's implementation (covered in detail in past reports) and presents early findings, 
based on two years of follow-up, on the difference that the program made in young people's 
educational attainment, employment, welfare receipt, and other important outcomes. A final 
report, based on approximately four years of follow-up, will present a more complete picture 
of program impacts and summarize findings on JOBSTART's cost-effectiveness. 
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The Goals of the JOBSTART Dcmoiistratiop and a Summary of Interim Findings 

Few programs have been shown to be effective in increasing the educational attainment, 
employment, and earnings of young people with poor skills. At the time JOBSTART was 
conceived, the evidence pointed toward more comprehensive programs, which attempt to 
remedy basic and occupational skills weaknesses, provide support services such as transportation 
and child care assistance, and substantially help participants find a job. JOBSTART was 
modeled after one of the few programs recognized to be a success story: the residential Job 
Corps. That program provides basic e^Jucation, occupational training, job placement assistance, 
and an extensive array of support services to the participating youths, who live at Job Corps 
Centers. 

However, the residential Job Corps cannot be offered to most young dropouts. It is 
relatively expensive, operates in specialized centers, requires the development of work 
experience positions with employers, and attracts only young people willina and able to live 
away from home. JOBSTART drew on the Job Corps' experience by offering most of the 
same basic components in a nonresidential program, although clearly the support services 
available in most demonstration sites were less extensive than those provided by the Job Corps. 
Also, there was no work experience component in JOBSTART. 

The JOBSTART Demonstration sought to answer four key policy questions relating to 
this general programmatic approach. 

• Recruitment. Could local agencies recruit young, economically 
disadvantaged) poorly skilled school dropouts? 

Many program operators have discovered how difficult it is to reach alienated young 
people and provide them with the hope pnd support they need to participate in an intensive 
program such as JOBSTART. Young school dropouts are often reluctant to return to a school 
setting, require extensive support services to participate, or seek immediate employment to 
meet pressing financial needs. In addition, the lengthy eligibility determination process 
characteristic of many programs (including JTPA's) may discourage some of those who are 
initially interested. 
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• Implementation. Could sites^ working within the funding and 
administrative constraints of JTPA, put in place a package of services 
that would address the needs of these youths? 

Sites participating in JOBSTART had to raise operating funds from existing programs, 
and most relied on Title IIA of JTPA. When the demonstration began (and to a considerable 
extent at the present time), federal and state regulations and prevailing administrative practices 
encouraged local JTPA administrative agencies (called service delivery areas) to emphasize 
shorter-term, lower-cost programs and to enroll participants who were more employable than 
the JOBSTART target group. More specifically, JTPA's performance standards (which 
emphasized the proportion of participants placed in a job, their wages, and the cost per 
"success story") created incentives to choose people who were more likely than the JOBSTART 
target group to achieve these successes at a relatively modest cost. All these factors led to a 
clear result: At the time JOBSTART began, only about one in four young people served 
under Title IIA of JTPA were school dropouts, and only about one in four of these school 
dropouts received basic education. Thus, the successful implementation of JOBSTART (with 
its relatively lengthy and intensive program components and its disadvantaged target group) 
could not be taken for granted within JTPA. 

• Participation. Would the young people respond favorably to this 
opportunity and make an investfiieni of their time and effort by 
participating in the services? 

Many youths need a substantial amount of education to improve their basic skills, and 
occupational training to instill job-related competencies, before they can be competitive in the 
job market. Yet the conditions that make it difficult to recruit them into education and 
training programs often preclude their completing the coursework. Financial pressures are 
severe, including the need for immediate cash to pay the rent and buy food. Since many 
disadvantaged young people work already - albeit sporadically and for very low wages - it is 
difficult to attract them into programs that cannot pay them stipends, as is the case under 
JTPA rules. Other barriers to participation - even in programs offering "free" training - 
include child care responsibilities, unstable housing arrangements, and peer pressure against 
participation. Programs must find ways to help "stabilize" the young people's lives so they can 
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move beyond their immediate problems and commit themselves to invest in learning new skills 
with a longer-term payoff. 

• Impacts. Would the program lead to an increase in educational 
attainment, and would this in turn have an impact on subsequent 
employment, earnings, and other outcomes? 

The JOBSTART Demonstration provided a rigorous test of the difference thai 
JOBSTARTs combination of nonresidential services made in the lives of young people. 
Youths who applied for the program were randomly assigned to a group given access to 
JOBSTART (the experimental group) or to a group not given that access but free to seek 
other services in the community (the control group). Since the two groups were created by 
chance, using a lottery-like process, there was only one systematic difference between them: 
Only those in the experimental group could receive JOBSTART services. Thus, the control 
group provides informatif jn on what those in the experimental group would have done if there 
had been no JOBSTART program: Some would have found alternative services, some would 
have worked, and so forth. Therefore, a comparison of the two group5;* behavior over time 
provides an estimate of the difference that the added services the experimental group received 
(that is, services above the level the control group received on its own) had on their 
subsequent employment, earnings, welfare receipt, and other outcomes. 

Findings available at this half-way point in the follow-up period provide answers to the 
first three questions and a partial answer to the fourth: 

• RecruUmefit* Through intensive outreach efforts, sites were able to recruit 
the target population of poorly skilled* economically disadvantaged young 
people, although several sites were unable to meet their recruitment 
goals. 

• Implementation. With considerable special effort, sites were able to fund 
and operate the program components within the JTPA system, and the 
program model received a fair test in most sites. 

• Participation. Participation by the experimental group in education and 
training was substantial and was much above that of the control group 
youths who sought similar types of services elsewhere. However, most 
youths fell short of the hours-of-attendance targets for the demonstration. 

• Impacts (on educational attainment). During the 24 months of follow-up 
available for this report, JOBSTART led to a doubling of the rate ot 
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receipt of a high school diploma or GED (high school equivalency 
credential) relative to the control group. This was true for both the full 
sample and most subgroups. 

Information on longer-run labor market outcomes is not yet available. The early findings 
may be summarized as follows: 

Early impacts on employment As expected^ more youths in the control 
group worked during the first year of follow-up than did those in the 
e;q>erimental group, for whom participation in JOBSTAJll was a major 
activity. In the second year of follow-up, approximately qual proportions 
of youths in the two groups worked. 

• Early impacts on earrings* During the first year of follow-up, when a 
large portion of the experimental were active in JOBSTART, controls* 
earnings^ not surprisingly, exceeded experimental' earnings by a 
significant amount. The average earnings of the experimental group 
remained below those of the control group during the second year. 
However, the gap in annual earnings showed an encouraging trend, 
narrowing from $585 in the first year to $205 in the second, and this 
second-year difference was not statistically significant. 

• Early impacts for key subgroups. There were early differences among 
key subgroups, although smaller sample sizes make conclusions less 
certain. Among men, earnings of experimental were significantly less 
than those of controls in both years of follow-up, although the difference 
narrowed. For women living with their own children and for other 
women, the trend was more favorable: After earning less than controls 
in the first year, experimental in both categories of women earned 
slightly more than controls in the second year. 

• Impacts for individual sites. Individual sites' impacts varied widely but, 
except for impacts on educational attainment, the differences across sites 
were not statistically significant. There was no clear relationship between 
individual sites' impacts and whether sites offered education before 
training or both services concurrently, or whether they provided ail 
services themselves or arranged for some services to be provided by 
other agencies. 

Later sections of this Executive Summary present these results in more detail, again 
grouping them under the topics of program recruitment, implementation, participation, and 
impacts. 



The Structure of the JOBSTART Demonstration 



The JOBSTART Demonstration guidelines specified the target group and the character 
of the core service components. The program was to target 17- to 21-year-old, economically 
disadvantaged school dropouts who read below the eighth-grade level and were eligible for 
JTPA Title IIA programs or the Job Corps (which is funded under Title IVB of JTPA). The 
four central program components were to be implemented as follows: 

* Instruction in basic academic skills was to be based on individualized 
curricula chosen by the sites to allow youths to proceed at their own 
pace toward competency goals in reading, communication, and basic 
a)mputational skills. 

* Occupational skills training was to be given in a classroom setting 
combining theory and hands-on experience to prepare participants for 
jobs in high-demand occupations* 

* Training-related support services were to includv-^ assistance with 
transportation and child care, counseling, and, where possible, additional 
support such as work-readiness and life skills (practical everyday 
knowledge) training and needs-based or incentive payments tied to 
program performance* 

* Job placement assistance was to be provided to help JOBSTART youths 
find training-related jobs. 

Sites were required to offer at least 200 hours of basic education and at least 500 hours of 
occupational training. These minimums were set after balancing two factors: (1) Youths in the 
JOBSTART target group were likely to need a substantial amount of education and training 
if they were to have real opportunities to become competitive in the job market, and (2) the 
administrative practices of JTTA made higher service targets unrealistic. 

Within this general framework, the 13 local JOBSTART programs did vary, reflecting 
their diverse operating experiences, funding sources, clientele, and local service networks. 
Among the important types of local variation were: additional program entry requirements in 
some sites, the extent of integration of education and training instruction, whether the young 
people were taught in separate classes or by being "mainstreamed" in classes with adults, the 
number and duration of occupational training courses (small community-based organizations 
typically offered far fewer courses than did other sites), and the strength of the implementation 
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of the core JOBSTART components, especially training and job placement assistance. 

The sites are listed in Table !• They included six community-based organizations, three 
adult vocational schools, a community college, and three Job Corps Centers that already 
operated nonresidential Job Corps programs. 

Findings on Recruitment of Youths 

Analysis of the characteristics of the 1,839 youths who made up the research sample for 
this report focused on three key questions: Did sites succeed in recruiting the demonstration's 
intended target group? Were these JOBSTART youths more or less disadvantaged than those 
served in other programs? Were there observed educational, employment, or other differences 
in the backgrounds of subgroups of youths (such as men, women living with their children, and 
other women) that might help explain differences in their participation and impacts? 

• With considerable effort^ the sites recruited the poorly skiilled, 
economically disadvantaged young people making up the intended target 
group for the demonstration. 

The youths in the demonstration were all eligible for JTPA Title IIA programs or the 
Job Corps; they included slightly more women than men. Most of the sample were merabers 
of minority groups and unmarried; nearly three-fourths A^ere under 20 years of age; almost one- 
half did not work during the year prior to random assignment; and about three-fifths left 
school before the eleventh grade. 

JOBSTART youths appear to have been more disadvantaged than the majonty of young 
people - or even young dropouts - served nationwide by JTPA Title IIA programs during the 
period JOBSTART was in operation. However, those in JOBSTART, especially the men, were 
not among the most disadvantaged youths (young people who rarely participate in any 
program) and were probably slightly more skilled than the typical residential Job Corpsmember 
during the same period. 

• Among JOBSTART youths, women living with their own children had 
noticeably weaker ties to employment than did other women or men* 

Prior work experience could affect both the likely behavior of the youths in the control 
group and the service needs of participants in the program. Women living with their own 
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TABLE 1 
THE JOBSTART SITES 



Agency Name 
and Location 


Type of 
Organization 


Prior Service 
Emphasis^ 


JOBSTART 

Program Structure'^ 


Allentown Youth Services 
Consortium, Buffalo, NY^ 


Community-based 


Education 


Sequential /brokered 


Atlanta Job Corps, 
Atlanta, GA 


Job Corps Center 


Education and 
training 


Concurrent 


Basic Skills Academy (BSA) , 
New York, NY 


Community*based 


Education 


Sequential /brokered 


Capitol Region Education 
Council (CREC), 
Hartford, CT 


Community-based 


Education 


Sequential /brokered 


Center for Employment 
Training (CET), 
San Jose, CA 


Communi ty-based 


Trai ning with 
some education 


Concurrent 


Chicago Conmons Association's 
Industrial and Business 
Training Programs, 
Chicago, IL 


Community-based 


Training 


Concurrent 


Connelley Skill Learning 
Center, Pittsburgh, PA 


Adult vocational 
school 


Education and 
training 


Concurrent 


East Los Angeles Skills 
Center, Monterey Park, CA 


Adul t vocational 
school 


Education and 
training 


Concurrent 


El Centro Conrounity College 
Job Training Center, 
Dallas, TX^ 


Community college 


Education and 
training 


Sequential /in-house 


Emily Griffith Opportunity 
School (EGOS) , Denver, CO 


Adult vocational 
school 


Education and 
training 


Concurrent 


Los Angeles Job Corps, 
Los Angeles, CA 


Job Corps Center 


Education and 
training 


Sequential /in-house 


Phoenix Job Corps, 
Phoenix, AZ 


Job Corps Center 


Education and 
training 


Concurrent 


SER/Jobs for Progress, 
Corpus Christi , TX 


Community-based 


Training 


Concurrent 



NOTES: ^Education refers to basic education, often as preparation for the GEO examination. Training 
refers to instruction in occupational skills needed for specific jobs. 

^Concurrent programs offer basic education and occupational training concurrently from the 
beginning of participation. Sequential/ in-house programs offer basic education followed by occupational 
training, with both components provided in-house by the agency. Sequential /brokered programs provide basic 
education and then serve as a broker for occupational training, referring participants to other agencies. 

^I^ October 1990 this site was renamed The Clarkson Center, Inc. 

^In September 1988 this site was renamed the Edmund J. Kahn Job Training Center. 
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children were least likely to have worked during the year before random as*;ignment and most 
likely to have received Aid to Families with Dependent Children (AFDC) and Food Stamps. 
Other women worked much more, and the men in the sample worked most of all Because 
of clear differences in initial characteristics between mothers caring for their children and other 
women, the gender-based subgroup analyses that follow divide the female sample into these 
two groups. However, in most of the analysis, all men are grouped together because only a 
small number of men reported at the outset that they were parents. 

Findings on Program Implementation 

• In general, the JOBSTART program model received a fair test in the 
demonstration. Most sites were able to put the program model in place, 
although the sites varied considerably in the intensity of their services 
and the way they were offered. 

Basic education, occupational training, support services, and job placement assistance 
were available to participants in each site, but the varying ways these services were linked 
reflected the sites' past operational experience and current capacity to provide services in- 
house. As shown in Table 1, eight cf the 13 sites provided >^asic education and occupational 
skills training concurrently ("concurrent" sites); two provided a sequence of education followed 
by training ("sequential/in-house" sites); and three provided education and then referred 
participants to other agencies for training f sequential/brokered" sites). In sites offering 
education and training concurrently, participants usually attended two hours of education 
classes and four hours of vocational training a day* In sites operating a sequential program, 
participants generally attended three hours a day of basic skills classes during the education 
phase, with the remaining three hours a day being devoted to life skills classes and 
employability workshops. As discussed later, participation rates and hours in the components 
of the model differed among these three types of sites. 

Basic Education Activities 

Sites successfully implemented the JOBSTART basic education 
component, although they varied in educational emphasis. 

The education component typically consisted of individualized instruction, which allowed 
students to move at their own pace learning reading, mathematics, and other subjects needed 
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to pass the General Educational Development (GED) examination. 

In sites where funding for education services was based on students passing the GED 
examination, staff made GED certification an important short-term goal and emphasized the 
skills tested on the GED examination in their education components. Two concurrent sites 
- CET/San Jose and Chicago Commons - provided some or all of their basic skills instruction 
as part of vocational training, with a focus on improving those skills needed to successfully 
complete the training. The remaining sites fell between these extremes* 

Occupational Skills TrainiM 

• Despite great variation in course offerings among the sites, JOBSTART 
youths generally studied occupations with skills requirements 
comparable to those for adults served within JTPA nationwide. 

Participants at large vocational schools could choose courses in more than 20 
occupations, and the Job Corps Centers and large community-based organizations also offered 
a wide range of training. In contrast, small community-based organizations providing training 
in-house typically offered fewer than five courses. In theory, youths ai sequential^rokered 
sites could choose courses from a variety of agencies, but in practice their choices were limited 
because they could not meet entrance requirements or encountered other administrative 
difficulties in gaining admission. Occupational choices for men and women followed traditional 
patterns, with about three-fourths of the women in clerical fields and slightly more than one- 
half of the men in machine trades, metal fabrication^ or construction. 

JOBSTART training was intended to prepare participants primarily for jobs requiring 
moderate skills (about one-half of participants) or higher skills (about one-fourth of 
participants). This distribution of skills ratings for training occupations was similar to what the 
U.S. General Accounting Office found for JTPA adult programs, an unexpected result since 
JOBSTART participants faced more barriers to employment than did the typical JTPA adult 
client. 

Training-Related Support Services 

• All sites provided transportation and child care assistance, but the 
availability of other services varied greatly. 

Clearly, the support services and other activities available at the Job Corps Centers 
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surpassed those at the other sites in both breadth and intensity. However, many sites were 
able to provide instruction in life skills and some type of needs-based payments. 

Job Placement Assistance 

• The job placement component of the program was the least developed 
in many sites. In particular, participants leaving JOBSTART before 
completing the curriculum received relatively little aid in finding a Job, 

In sites with strong job placement assistance, instruction in proper work behavior, 
employer expectations, and job search techniques began while students were still in training; 
placement specialists provided leads and assistance in fmding jobs; and staff had especially 
strong ties to local employers. Job placement assistance was noticeably weaker in the three 
sequential/brokered sites, where it was the responsibility of the training agency. Unfortunately, 
most participants never reached the training components, so only informal assistance was 
available from the JOBSTART agency. 

Findings on Participation in JOBSTART 

• Young people in the experimental group attended an average of more 
than 400 hours of group activities, which is impressive when compared 
to the experience of many other JTPA-funded programs. Despite this 
relative success, most young people did not acquire the skills needed to 
pass the GED examination and did not participate enough to complete 
an occupational trainLig course* 

Experimentals (including the 11 percent who did not participate at all) averaged 415 
hours of attendance in JOBSTART classes, as shown in Table 2. The young people spent 
most of this time in education and occupational training, as opposed to other activities such 
as life skills training. Slightly more than one-third of all experimentals participated for more 
than 500 hours, and another one-quarter were active for 201 to 500 hours. These findings 
show that JOBSTART succeeded in engaging more than one-half of the youths in the 
experimental group in the program and its activities, but for about two-fifths of those in the 
experimental group, participation was neither lengthy nor intensive. 

The average length of stay in JOBSTART was 6.8 months, with 16 percent of 
experimentals still active in the program 12 months after random assignment and nearly 10 
percent active 15 months after random assignment. This means that for most experimentals 



-xvii- 



TABLE 2 



PARTICIPATION RATES, HOURS OF PARTICIPATION, AND LENGTH OF STAY 

FOR EXPERIMENTALS 



Activity Measure Experimental s 

Percent participating in 

Any activity 88,7 

Education 85.9 

Training 66.6 

Education and training 64.4 

Other activities 40,0 

Average hours in 

Education 128,1 

Training 248,9 

Education and training 377.0 

Other activities 37,3 

A*^! activities 414,8 

Percentage distribution of hours 

in education and training 

None 11,9 

Up to 200 33,2 

201 to 500 22,4 

501 to 700 15,5 

701 or more 17,0 

Total 100,0 

Percentage distribution of hours 
in all activities 

None 11,3 

Up to 200 28.6 

201 to 500 25,5 

501 to 700 15,2 

701 or more 19.5 

Total 100,1 

Length of stay (months) 

Average 6,8 

Median 6,0 

Percent still participating in month 

3 78.0 

6 53,6 

9 30,6 

12 16,4 

15 9.6 

18 4,8 

19 or later 3.7 

Number of experimental s 949 

NOTES: Calculations for this table used data for all experimental for 
whom there were 24 months of follow-up survey data, including those with values 
of zero for outcomes and those who were assigned to JOBSTART but did not 
parti cipate. 

Distributions may not total 100.0 percent because of rounding. 
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the first year of follow-up was primarily a period of program participation and that for 16 
percent the second year also included months with program activity. JOBSTART 
experimentals stayed in the program considerably longer than the typical young dropout served 
in JTPA Title IIA programs and nearly as long as the average Job Corpsmember. 

• Differences in participation were associated x^th choices made at the 
site level (1) to offer concurrent versus sequential education and 
training and (2) to house all services on-site nether than refer youths 
elsewhere for training* 

Youths in sequential/in-house sites averaged the highest total hours, followed by youths 
in concurrent programs. Those in sequential/brokered sites averaged noticeably fewer total 
hours, primarily because only about 25 percent of the young people successfully made the 
transition to the off-site occupational training. This occurred because of difficulties arranging 
linkages with another organization for training within JTPA and because youths at these 
education-oriented sites may have been more interested in studying for their GED credential 
than in getting occupational training. (Young people in sequential/brokered sites averaged the 
most education hours,) As would be expected, sequential sites showed a higher proportion of 
experimentals still active in the program a year after random assignment. 

• Participation was surprisingly similar among key demographic groups. 

Men, women living with their own children, and other women averaged 410, 408, and 
429 total hours, respec.ivwiy. The groups* average lengths of stay in the program were quite 
similar, although a somewhat higher proportion of women than men were still active in 
JOBSTART 12 months after random assignment. Youths who had not been arrested between 
their sixteenth birthday and the time of random assignment did have significantly higher 
average hours of participation. However, other subgroup analysis (such as by race, age, grade 
level at school dropout, marital status, and receipt of public assistance) showed no statistically 
significant differences in average total hours. 

Findings on Program Impacts 

Young people in the JOBSTART sample could have improved their skills by several 
means: through participating in JOBSTART (for those in the experimental group) or in other 
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education and training programs^ or by learning on the job. Program impacts are measured by 
comparing the experiences of the experimental and control groups; in essence, this compares 
the payoff of the investment made by experimental with that made by controls. 

A tVamework for Aiialvzin£ JOBSTARPs L^bor Market Impacts 

As is true of most investments, many of the costs of participating in a program such as 
JOBSTART are incurred in the short run. The benefits will accrue over the young people's 
lifetime if they learn new skills that pay off in the labor market. Figure 1 presents a 
theoretical framework for analyzing the labor market aspects of this investment and the 
alternative investments made by young people in the control group. Those in JOBSTART 
committed their time and effort to improving their skills in the expectation of a future payoff. 
While participating, they gave up the chance to work and earn, so any forgone -ncome was an 
in-program opportunity cost of the program, represented in the figure by the shaded area 
between the start and end of JOBSTART participation. 

The figure also shows the control group's earnings rising over time. This reflects their 
growing employment rate and, for some, the acquisition of new skills on the job. For young 
people with poor skills, work experience can be an important source of new skills, which can 
translate into increased productivity and earnings and more stable employment. Once their 
participation in JOBSTART ended, young people in the experimental group will have looked 
for jobs, but their employment rates and earnings may not immediately exceed (or even reach) 
those of controls who were already working rather than attending a progr am. This post- 
program opportunity cost is the area in the figure between the end of program participation 
and the hoped-for point at which the earnings of the experimental group exceed those of the 
control group. 

For a program such as JOBSTART to pay off for young people, the long-term benefits 
of increased education and training (represented in the figure by the shaded area on the right) 
must exceed the forgone - more immediate - rewards of possible earnings and enhanced 
skills through work experience. Even in successful programs, it will take time for participants 
to overcome the head start of those who have been working throughout the program period. 
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FIGURE 1 



A THEORETICAL VIEW OF THE PAYOFF 
OF A PERSONAL INVESTMENT IN EDUCATION AND TRAINING 
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The Two-Year Impact Sample 

The 2,312 youths who applied lor JOBSTART and were judged eligible were randomly 
assigned to either the experimental or control group. Follow-up surveys attempted to reach 
all members of both groups 12 and 24 months after they were randomly assigned* This analysis 
of program impacts uses a sample of 1,839 (80 percent of all those who were randomly 
assigned). These are the youths who provided information for this two-year period. Impacts 
reported in the following sections were statistically significant (that is, unlikely to have arisen 
by chance) unless othenvise noted. 

ExperimeRml-Control Differences in Participation in Education and Training 

For JOBSTART to make a difference in the lives of the young people, those in the 
experimental group must have participated in substantially more education and training 
activities than did those in the control group, who had access to other services in the 
community. 

• During the follow-up period, experimentals were nearly twice as likely 
as controls to have partiripated in some type of education or training. 
However, nearly one-half of all controls also got some education or 
training. 

Table 3 shows experimental-control differences in the percentage of the youths who ever 
participated in education and training; the differences are shown for both the full sample and 
key subgroups. In the two years following random assignment, 93 percent of experimentals 
versus 44 percent of controls received some type of education or training. Experimentals 
averaged 619 hours in these activities, whereas controls averaged 250 hours. The differences 
were largest during the early months of the foUow-up period, when most experimentals were 
active in JOBSTART, and gradually disappeared by the end of the two years. All the 
subgroups that were analyzed showed large differences between experimentals and controls 
in service receipt. There was a greater experimental-control difference in the percentage of 
youths participating in education than in the percentage participating in training, because many 
young people at sequential (and especially sequentiaVbrokered) sites did not participate in 
training. 
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TABLE 3 



IMPACTS OF JOBSTART ON SERVICE RECEIPT AND EDUCATIONAL ATTAINMENT 
DURING THE 24 MONTHS FOLLOWING RANDOM ASSIGNMENT 



Outcome and Subgroups 


Sample Size 


Experimental 


Control s 


Difference 


Ever received any education 










or training 










Full sample 


1,839 


92.7V 


44.2% 


48.4*** 


Men 


871 


92.8 


36.6 


56.1*** 


Women living with 










own child(ren) 


484 


93.6 


47.9 


45.7*** 


Women not living with 










own child(ren)* 


484 


92.5 


54.1 


38.4*** 


Received GED or high school 










diploma 










Full sample 


1,839 


33.2 


16.4 


16.7*** 


Men 


871 


32.1 


16.9 


15.2*** 


Women living with 










own child(ren) 


484 


35.5 


14.2 


21.3*** 


Women not living with 










own child(ren)* 


484 


32.4 


18.2 


14.2*** 



NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

Statistical significance levels are indicated as = 1 percent; * 5 
percent; * = 10 percent. 

^Includes women who did not have children. 
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Impacts on Educational Attainment 



• JOBSTART led to a doubling of the rate of GED ct^rtiflcation or receipt 
of a high school diploma, from 16.5 percent of the control group to 33.1 
percent of the experimental group. 

Table 3 presents JOBSTARTs impacts on educational attainment during the 24 months 
of follow-up and includes both the full sample and key subgroups. The full sample impact on 
attainment of a GED or high school diploma '^as 16.7 percentage points, similar to the results 
found in an evaluation of the residential Job Corps. 

• These large educational attainment impacts were present for many 
diffTerent subgroups in the overall sample. 

Among men, 32.1 percent of experimentals versus 16.9 percent of controls completed high 
school or passed the GED examination during the follow-up period, for an impact of 15.2 
percentage points. For women living with their own children, the figures were 35.5 percent 
of experimentals and 14.2 percent of controls, for an impact of 21.3 percentage points; and for 
other women, the figures were 32.4 percent of experimentals and 18.2 percent of controls, for 
an impact of 14.2 percentage points. Numerous other subgroups based on work experience, 
welfare receipt, prior education, initial reading level, and age all showed similar large impacts. 

Impacts on Employment and Earnings for the Full Samp le 

• As expected, more youths in the control group than in the experimental 
group worked during the first year of follow-up. In the second year of 
ioUow-up, the proportions were not significantly different 

As shown in Table 4, 57.6 percent of experimentals and 61.6 percent of controls worked 
at some time during the first year of follow-up, for a 4.0 percentage point decrease in 
employment among experimentals relative to controls. At some point during the second year, 
69.5 percent of controls and 72.0 percent of experimentals worked. This 2.5 percentage point 
impact was not statistically significant The control group's employment rate exceeded the 
experimental group's by the largest amount in month 5, after which the difference narrowed. 
In the second year of follow-up, the experimentals' employment rate exceeded the controls' 
rate in each of six months and was below it in the other six months; however, in no month was 
this difference statistically significant 
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TABLE 4 



IMPACTS OF JOBSTART ON EMPLOYMENT AND EARNINGS 
DURING THE 24 MONTHS FOLLOWING RANDOM ASSIGNMENT 



Subgroup, Outcome, 
and Foliow-Up Period 



Experimental s 



Controls 



Difference 



Full sample 

Ever employed (%) 

Months 1-24 

Months 1-12 

Months 13-24 
Total earnings ($) 

Months 1-24 

Months 1-12 

Months 13-24 
Sample size 

Hen 

Ever employed (%) 

Months 1-24 

Months 1-12 

Months 13-24 
Total earnings ($) 

Months 1-24 

Months 1-12 

Months 13-24 
Sample size 

Uomen living with own child(ren) 



79.2 
57.6 
72.0 

5,859.56 
1,965.47 
3,894.08 
949 



89.0 
66.7 
84.3 

7,797,22 
2,648.50 
5,148.72 
438 



77.9 
61.6 
69.5 

6,649.07 
2,550.26 
4,098.81 
890 



90.1 
74.8 
84.3 

9,492.61 
3,676.88 
5,815.73 
433 



1.3 
-4.0* 
2.5 

■789. 52*** 
■584. 78*** 
■204.73 



-1.1 
-8.1*** 
0.0 

■1,695.39*** 
■1,028.38*** 
-667.01* 



Ever employed (%) 








Months 1-24 


62.1 


57.2 


4.9 


Months 1-12 


41.0 


38.3 


2.7 


Months 13-24 


53.3 


46.1 


7.2 


Total earnings ($) 








Months 1-24 


3,035.55 


2,952.31 


83.24 


Months 1-12 


1,011.83 


1,100.96 


-89.13 


Months 13-24 


2,023.73 


1,851.35 


172.38 


Sample size 


250 


234 




omen not living with own child(ren)'^ 








Ever employed (%) 








Months 1-24 


79.3 


75.8 


3.4 


Months 1-12 


57.0 


61.5 


-4.5 


Months 13-24 


69.5 


65.3 


4.2 


Total earnings ($) 








Months 1-24 


5,070.23 


5,290.80 


-220.58 


Months 1-12 


1,607.50 


2,030.02 


-422.53* 


Months 13-24 


3,462.73 


3,260.78 


201.95 


Sample size 


261 


223 





NOTES: Calculations for this table used data for all sample members for whom 
thert were 24 months of follow-up survey data» including those with values of zero for 
outcomes and those who were assigned to JOBSTART but did not participate. 

Statistical significance levels are indicated as *** = 1 percent; ** = 
5 percent; * = 10 percent. 

^Includes women who did not have children. 



• Experimentals earned significantly less than controls in the first year 
of follow-up. In the second year, the gap between experimentals and 
controls narrowed, and the difference was no longer statistically 
significant 

As expected, experimentals earned less than controls during the first year of follow-up 
(see Table 4); this $585 difference was a clear opportunity cost of participating in the program. 
In the second year, although the proportion of experimentals working drew even with the 
proportion of controls, experimentals continued to lag slightly (but not significantly in the 
statistical sense) behind controls in hours worked per week and weeks worked per month. As 
a result, the earnings of experimentals remained below those of controls during the second 
year; however, the difference ($205) was no longer statistically significant. The cumulative 
opportunity cost in the form of forgone earnings was, therefore, $790. 

Impacts on Employment and Earnings for Key Suberoups 

Many past studies of nonresidential education and training programs have found starkly 
different results for men, women living with their children, and other women. Thus, it is 
important to move behind the findings for the full sample and examine subgroups that are of 
special policy interest. In this analysis, sample sizes arc smaller and the conclusions are 
therefore less certain. 

• There were early differences among key subgroups. For men, earnings 
impacts were negative throughout the two-year follow-up period, while 
kor women there were signs of a favorable trend. 

Table 4 shows employment rate and earnings impacts for these subgroups. Among men, 
a significantly lower percentage of experimentals than controls worked at some point in the 
first year, but in the second year employment rates were equal. Earnings for experimentals 
were significantly below those of controls in both years, although the gap did narrow. For the 
two groups of women, the pattern over time was more positive, but sample sizes were smaller, 
so almost all of the impacts were not statistically significant. Among women living with their 
own children at the time of random assignment, a higher percentage of experimentals than 
controls worked in each of the two years, with the second year showing a somewhat larger 
employment rate impact. For this subgroup, controls earned slightly more than experimentals 
during the first year, while experimentals earned more during the second, resulting in a small 
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positive earnings impact for the two-year period. For other women, a higher percentage of 
controls than experimentals worked at some point in the first year, while in the second year, 
a higher percentage of experimentals worked. Reflecting this, the earnings of controls 
exceeded those of experimentals (by a statistically significant amount) in the first year; in the 
second year, experimentals' earnings were higher than controls' earnings, but the two-year total 
for earnings was slightly negative. 

• The employment experience of the controls in the three groups provides 
much of the explanation for the pattern of impacts among the groups* 

One likely explanation for better employment results for women is that it is easier to 
improve the employment and earnings of those who do not spend much time in the woild of 
work (for example, young mothers) than of those who are already in the labor force but f2.ii 
to find and keep steady, well-paying jobs (for example, poorly skilled young men). Thus, from 
this perspective, women have greater potential to improve their labor market outcomes than 
do men, and less to lose (in terms of forgone employment and earnings) by investing in 
education and training. 

The JOBSTART sample followed this pattern, as shown in Table 4. During the first 
year after random assignment, 74.8 percent of control men worked at some point, compared 
to 38.3 percent of women living with their own children and 61.5 percent of other women. 
During this period, which, for many experimentals, consisted mostly of program participation, 
the impact of JOBSTART on employment rates was 2.7 percentage points for the young 
mothers, -4.5 percentage points for other women, and -8.1 percentage points for men. Men, 
and women who were not caring for children, thus paid an opportunity cost for JOBSTART 
in terms of forgone employment, while young mothers did not. 

• The occupational distribution of training for men and women in 
JOBSTART may have also contributed to these differences in impacts. 

A second possible explanation for the poorer impacts observed for men in JOBSTART 
is the greater difficulty of placing men in jobs that reward a GED. Many women in 
JOBSTART, for example, were trained in clerical occupations and sought that type of work, 
a domain in which educational credentials are important. However, young men were more 
likely to find work in occupations that did not reward the GED, at least not in the early stages 
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of employment. Many types of blue-collar work, especially physically demanding work, may 
well fall into this category. The problem for men could be further exacerbated by the greater 
difficulty of finding training-related employment for men who do study occupations in which 
having a GED matters. 

Impacts on Other Outcomes for the Full Sample 

• During the first 24 months of follow-up, JOBSTART had no statistically 
significant impacts on a variety of other outcomes* 

This general conclusion was true for the full sample and both women and men, and 
applies to a range of outcomes including receipt of most public benefits, childbearing, fathering 
of children, provision of child support (by noncustodial parents), and crimf'^al arrests. The 
lack of impacts on criminal arrestr^ differs from the finding in a study of the residential Job 
Corps program, but there is an obvious reason to expect much less of an early impact on 
criminal behavior for JOBSTART In the residential Job Corps program, young people moved 
from their own communities to special centers that provided the education and training 
services. Many of these centers are in isolated areas or in communities without large gang 
populations or heavy involvement of youths in the drug trade. Much of the Job Corps' impact 
on criminal behavior came during the in-program period because of this '^isolation** effect. In 
JOBSTART, no such change took place in the young people's lives; they continued to live in 
their own neighborhoods and to spend time outside the program with their existing circle of 
friends. 

Site Differences in Impacts 

There is strong policy interest in the influence of program characteristics on impacts. 
However, it is extremely difficult to draw clear lessons from differences in the impacts observed 
for different types of JOBSTART sites. This study was not designed ;o analyze how variations 
in the way the basic JOBSTART model was implemented may have affected impacts. Youths 
were randomly assigned to the experimental or control group in each site, rather than to 
different types of programs, which would be necessary for a rigorous experiniental test of the 
effectiveness of alternative approaches. 

Attempts to use differences in impacts among the sites to understand the influence of 
program characteristics face serious problems. The number of sites was limited. Sample sizes 
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in the individual sites were small, so most experimental-control differences within a site were 
not statistically significant. Furthennore, when the impacts in each site are compared, the 
differences among the site-specific impacts are not significant on most outcomes* Finally, the 
sites differed in many characteristics including the background of the youths in the sample, 
labor market conditions, the level of alternative services received by controls, and many 
dimensions of their JOBSTART programs. 

Despite these difficulties, if a clear pattern of impacts across the sites emerged, it migh: 
still be possible to draw some tentative conclusions. For example, if most sites operating 
sequential programs of education followed by training had impacts that differed markedly from 
those of sites offering education and training concuirently from the start of program 
participation, one might conclude that this difference in program structure was an important 
factor in understanding the differences in impacts. 

* There was no clear pattern of impacts among the 13 sites that supports 
conclusions about the effect of program characteristics on program 
impacts. 

This general point can be illustrated using site-specific impacts on second-year earnings. 
One aspect of program design that is of special interest is the choice of sequential programs 
of education followed by training versus concurrent programs. When the JOBSTART sites are 
divided into two groups based on this program feature, within each group there are sites with 
both positive and negative earnings impacts. When the sequential programs are further 
subdivided into those offering all services themselves and those linking up with other agencies 
for some program components, within each of these groups there are both positive and 
negative impacts. To sum up the site story on this outcome: Ve^y different types of programs 
had positive impacts. One program >vith positive earnings impacts was oriented toward training 
and offered education primarily in the context of occupational training. This program also had 
a strong job placement effort, little initial screening of applicants, and a relatively short length 
of participation. A second, in contrast, placed a heavy emphasis on basic education, and only 
one-third of its participants were active in any occupational training. It had a relatively weak 
job placement effort and lengthy participation. A third program, between these two extremes, 
also had positive impacts on second-year earnings. 
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Next Steps in the Research 

The JOBSTART Demonstration tackled a difficult issue, the employment problems of 
poorly skilled, economically disadvantaged young people. This report provides early follow-up 
on how a program of education and trainf' g, support services, and job placement assistance will 
affect their employment and earnings. When these young people entered the demonstration, 
most were teenagers. The four-year follow-up survey, now in progress, will carry the 
JOBSTART story to the youths' early twenties, the time when most young people begin to 
apply their skills in a more serious and committed way to employment. 

The program impacts reported here compared two different kinds of skills-enhancing 
investments. The experimental group, most of whom invested in JOBSTART, paid a clear 
initial cost in forgone emplo>Tnent in the hope of future payoffs in employment and earnings. 
Although some members of the control group were in education and training programs, 
controls were more likely than experimental to be working; many were, in the process, 
learning new skills on the job. For these controls, the initial cost in forgone income was small 
or nonexistent. A long follow-up period is needed for a valid comparison of these two very 
different types of investments; if the follow-up is too short, the large initial costs of 
JOBSTART will be included, but not the chance to see if the payoff appears over time. Of 
course, as the long search for effective programs for disadvantaged young people illustrates, 
there is no guarantee that the payoff will occur. 

Lxjnger follow-up could be especially important in addressing two issues. Sites differed 
in the extent to which they emphasized education versus occupational training within 
JOBSTART. Training is more likely than education to have an immediate payoff if placement 
in a training-related job can be arranged. Bui education might have longer-run payoffs if it 
produces the basic skills needed to learn new skills in the future. The longer follow-up will 
provide an opportunity to understand the terms of this likely trade-off in program emphasis. 

Longer follow-up may also prove useful in helping to explain the clear difference in 
short-term impacts for men and women. Young men, especially minority men, appear to have 
a more difficult time than women in moving into jobs that reward better basic skills and that 
relate to their occupational training. Longer-term follow-up will provide a chance to see 
whether, over time, they can make this transition. 



-XXX 

ERIC 



CONTENTS 



PAGE 



ACKNOWLEDGMENTS iii 

PREFACE V 

EXECUTIVE SUMMARY vii 

TABLES AND HGURES xxxiii 

ABBREVIATIONS xxxviii 

CHAPTER 

1 AN OVERVIEW OF THE JOBSTART DEMONSTRATION 1 

I. The Youth Employment Problem and the Challenge 

of Serving Young School Dropouts 5 

II. The Changing Programmatic Context 

of the JOBSTART Demonstration 12 

III. The JOBSTART Demonstration 16 

IV. The JOBSTART Evaluation and the 

Organization of This Report 27 

2 STUDY DESIGN AND SAMPLE CHARACTERISTICS 30 

I. An Overview of the Study Design 30 

II. The Characteristics of the JOBSTART Youths 38 

3 .<\N OVERVIEW OF JOBSTART SERVICES AND PARTICIPATION ... 57 

I. The Nature of JOBSTART Services 58 

II. The Intensity of JOBSTART Participation 75 

III. Moving Behind the Aggregate 

Participation Measures 81 

IV. Summary of the Features of the Sites 93 

4 RECEIPT OF EDUCATION AND SKILLS TRAINING 

AND JOBSTARTS IMPACTS ON EDUCATIONAL ATTAINMENT 96 

I. Receipt of Education and Training 

by Experimentals and Controls 97 

II. Impacts on Educational Attainment 114 



-xxxi- 

ERIC .^> 



PAGE 



5 IMPACTS ON EMPLOYMENT, EARNINGS, 

AND OTHER OUTCOMES 125 

I. JOBSTART as an Investment 126 

II. JOBSTARTs Impacts on Employment Rates 128 

III. JOBSTARTs Impacts on Earnings 141 

IV. Moving Behind These Employment-Related Impacts 144 

V. JOBSTARTs Impacts on Positive Activity 154 

VI. JOBSTARTs Impacts on Other Outcomes 155 

6 SITE VARIATION IN IMPACTS 161 

I. A Framework for Analyzing Site 

Variation in Impacts 164 

II. Full Sample Impacts, by Site 174 

III. Subgroup Impacts, by Site 179 

IV. Summary and Future Impacts To Watch 181 

APPENDICES 

A. Data Sources for the Evaluation 185 

B. Impacts of JOBSTART: 

Methodological Issues 195 

C. Supplemental Tables to Chanter 2 215 

D. Supplemental Table to Chapter 3 227 

E. Cost of the JOBSTART Program 237 

F. Supplemental Tables to Chapters 4 and 5 251 

G. Supplemental Tables to Chapter 6 261 

REFERENCES 271 

SELECTED MDRC PUBLICATIONS ON YOUTH PROJECTS 275 



-xxxii- 

ERIC 



TABLES AND FIGURES 



TABLE PAGE 

1 THE JOBSTART SITES xiv 

2 PARTICIPATION RATES, HOURS OF PARTICIPATION, 

AND LENGTH OF STAY FOR EXPERIMENTALS xviii 

3 IMPACTS OF JOBSTART ON SERVICE RECEIPT 

AND EDUCATIONAL ATTAINMENT DURING THE 24 MONTHS 

FOLLOWING RANDOM ASSIGNMENT xxiii 

4 IMPACTS OF JOBSTART ON EMPLOYMENT 

AND EARNINGS DURING THE 24 MONTHS FOLLOWING 

RANDOM ASSIGNMENT xxv 

LI LABOR FORCE STATUS AND EMPLOYMENT RATES 
FOR 16- TO 24-YEAR-OLDS NOT ENROLLED 

IN SCHOOL IN 1990 6 

1.2 THE JOBSTART SITES 18 

1.3 THE JOBSTART PROGRAM GUIDELINES 21 

2.1 CHARACTERISTICS AT THE TIME OF RANDOM 

ASSIGNMENT BY SITE 40 

2.2 CHARACTERISTICS AT THE TIME OF RANDOM 

ASSIGNMENT, BY PROGRAM STRUCTURE 49 

2.3 CHARACTERISTICS AT THE TIME OF RANDOM 

ASSIGNMENT, BY GENDER AND PARENTAL STATUS 53 

3.1 CHARACTERISTICS OF JOBSTART ACnVITIES, 

BY SITE 59 

3.2 PERCENTAGE DISTRIBUTION OF OCCUPATIONS 

FOR PARTICIPANTS IN TRAINING, BY GENDER 65 

3.3 BASIC SUPPORT SERVICES AVAILABLE 

IN JOBSTART, BY SITE 68 

3.4 RATINGS OF THE IMPLEMENTATION 

OF JOBSTART COMPONENTS, BY SITE 74 

3.5 PARTICIPATION RATES, HOURS OF PARTICIPATION, 

AND LENGTH OF STAY FOR EXPERIMENTALS 76 

-xxxiii- 



ERIC 



TABLE 



PAGE 



3.6 PARTICIPATION AND LENGTH OF STAY FOR YOUNG 

DROPOUTS IN JTPA TITLE IIA, BY ACTIVITY 79 

3.7 PARTICIPATION RATES, HOURS OF PARTICIPATION, 

AND LENGTH OF STAY, BY GENDER AND PARENTAL STATUS ... 82 

3.8 AVERAGE TOTAL PARTICIPATION HOURS, BY SELECTED 
CHARACTERISTICS OF EXPERIMENTALS AT THE TIME 

OF RANDOM ASSIGNMENT 85 

3.9 PARTICIPATION RATES, HOURS OF PARTICIPATION, 

AND LENGTH OF STAY FOR EXPERIMENTALS, BY SITE 87 

3.10 PARTICIPATION RATES, HOURS OF PARTICIPATION, 

AND LENGTH OF STAY, BY PROGRAM STRUCTURE 90 

3.11 SUMMARY OF JOBSTART IMPLEMENTATION, BY SITE 94 

4.1 IMPACTS ON RECEIPT OF EDUCATION OR TRAINING 

THROUGH MONTH 24 99 

4.2 IMPACTS ON TOTAL HOURS OF EDUCATION OR TRAINING 

RECEIVED THROUGH MONTH 24 102 

4.3 IMPACTS ON RECEIPT OF EDUCATION OR TRAINING 

THROUGH MONTH 24, BY GENDER AND PARENTAL STATUS .... 105 

4.4 IMPACTS ON RECEIPT OF EDUCATION OR TRAINING 
THROUGH MONTH 24, BY SELECTED CHARACTERISTICS 

AT THE TIME OF RANDOM ASSIGNMENT 108 

4.5 IMPACTS ON EDUCATIONAL ATTAINMENT 

THROUGH MONTH 24 116 

4.6 IMPACTS ON EDUCATIONAL ATTAINMENT THROUGH MONTH 

24, BY GENDER AND PARENTAL STATUS 117 

4.7 IMPACTS ON GED ATTAINMENT THROUGH MONTH 
24, BY SELECTED CHARACTERISTICS AT THE TIME 

OF RANDOM ASSIGNMENT 120 

5.1 IMPACTS ON EMPLOYMENT RATES 

THROUGH MONTH 24 129 

5.2 IMPACTS ON EMPLOYMENT RATES THROUGH MONTH 

24, BY GENDER AND PARENTAL STATUS 132 



-xxxiv- 



TABLE 



PAGE 



5.3 IMPACTS ON YEAR-TWO EMPLOYMENT RATES, 
BY SELECTED CHARACTERISTICS AT THE TIME 

OF RANDOM ASSIGNMENT 135 

5.4 IMPACTS ON EARNINGS THROUGH MONTH 24 142 

5.5 IMPACTS ON EARNINGS THROUGH MONTH 24, 

BY GENDER AND PARENTAL STATUS 145 

5.6 IMPACTS ON YEAR-T>\'0 EARNINGS, BY SELECTED 
CHARACTERISTICS AT THE TIME OF RANDOM 

ASSIGNMENT 147 

5.7 IMPACTS ON WEEKS EMPLOYED PER MONTH 

AND HOURS WORKED PER WEEK THROUGH MONTH 24, 

BY GENDER AND PARENTAL STATUS 152 

5.8 IMPACTS ON CRIMINAL ARRESTS THROUGH MONTH 

24, BY GENDER AND PARENTAL STATUS 159 

5.9 IMPACTS ON PREGNANCY AND CHILDBIRTH 
THROUGH MONTH 24, BY PARENTAL STATUS AT THE TIME 

OF RANDOM ASSIGNMENT 160 

6.1 SUMMARY OF JOBSTART IMPLEMENTATION, BY STTE 165 

6.2 EXPERIMENTAL-CONTROL DIFFERENCE IN SERVICE 

RECEIPT, BY STTE 171 

6.3 SELECTED IMPACTS OF JOBSTART, BY SITE 

(NOT ADJUSTED FOR SITE DIFFERENCES IN SAMPLE 
CHARACTERISTICS AT THE TIME OF RANDOM 

ASSIGNMENT) 175 

6.4 SELECTED IMPACTS OF JOBSTART, BY STTE 
(ADJUS'TED FOR SITE DIFFERENCES IN SAMPLE 
CHARACTERISTICS AT THE TIME OF RANDOM 

ASSIGNMENT) 176 

Al ACTIVITIES INCLUDED IN PARTICIPATION HOURS, 

BY SITE AND COMPONENT 188 

A2 RESPONSE RATES FOR 24-MONTH SURVEY, BY SITE 191 

B.l CHARACTERISTICS AT THE TIME OF RANDOM 

ASSIGNMEN'T, BY RESEARCH STATUS 196 

B.2 ESTIMATED REGRESSION COEFFICIENTS 
FOR THE PROBABILTTY OF ASSIGNMENT 

TO THE EXPERIMENTAL GROUP 199 

-XXXV- 



er|c :] 7 



TABLE PAGE 

B.3 ESTIMATED REGRESSION COEFnCIENTS FOR THE 

PROBABILITY OF UNIT SURVEY RESPONSE 203 

B. 4 ESTIMATED REGRESSION COEFFICIENTS 

FOR SELECTED OUTCOMES 208 

C. l CHARACTERISTICS IN THE YEAR BEFORE RANDOM 

ASSIGNMENT, BY EMPLOYMENT STATUS 217 

C.2 CHARACTERISTICS IN THE YEAR BEFORE RANDOM 

ASSIGNMENT, BY GENDER AND EMPLOYMENT STATUS 220 

C. 3 CHARACTERISTICS AT THE TIME OF RANDOM 

ASSIGNMENT, BY AGE 223 

D. l RATINGS AND SPECL\L FEATURES OF THE IMPLEMENTATION 

OF JOBSTART COMPONENTS, BY SITE 229 

E. l AVERAGE MONTHLY AND TOTAL COST^ 

PER EXPERIMENTAL AND PER PARTICIPANT, BY SITE 242 

E.2 AVERAGE JOBSTART OPERATING COSTS 

PER EXPERIMENTAL, BY SITE 247 

E. 3 AVERAGE JOBSTART SUPPORT SERVICES 

AND PARTICIPANT PAWENT COSTS PER EXPERIMENTAL, 

BY SITE 249 

F. l IMPACTS ON MONTHLY RECEIPT OF EDUCATION 

OR 1 RAINING 253 

F.2 IMPACTS ON MONTHLY AVERAGE HOURS 

OF EDUCATION OR TRAINING RECEIVED 254 

F.3 IMPACTS ON MONTHLY RECEIPT OF BASIC 

EDUCATION 255 

F.4 IMPACTS ON MONTHLY RECEIPT OF TRAINING 256 

F.5 IMPACTS ON MONTHLY EMPLOYMENT RATES, 

BY GENDER AND PARENTAL STATUS 257 

F. 6 IMPACTS ON MONTHLY RATES OF POSmVE 

ACTlvrTY, BY GENDER AND PARENTAL STATUS 259 

G. l EXPERIMENTAL-CONTROL DIFFERENCE 

IN SERVICE RECEIPT, BY SITE 263 

G.2 IMPACTS ON YEAR-TWO EMPLOYMENT RATES 

AND EARNINGS FOR MEN, BY STTE 264 

-xxxvi- 



ERIC 



TABLE 



PAGE 



G.3 IMPACTS ON YEAR-TWO EMPLOYMENT RATES 

AND EARNINGS FOR WOMEN LIVING WITH THEIR OWN 
CHILD(REN) AT THE TIME OF R.\NDOM 

ASSIGNMENT, BY SITE 266 

G.4 IMPACTS ON YEAR TWO EMPLOYMENT RATES 

AND EARNINGS FOR WOMEN NOT LIVING WITH THEIR 
OWN CHILD(REN) AT THE TIME OF RANDOM 

ASSIGNMENT, BY SITE 268 

FIGURE 

1 A THEORETICAL VIEW OF THE PAYOFF 

OF A PERSONAL INVESTMENT IN EDUCATION 

AND TRAINING xxi 

2.1 THE JOBSTART EVALUATION DESIGN 

AND SAMPLE R.OW 32 

4.1 MONTHLY RECEIPT OF EDUCATION OR TRAINING, 

BY RESEARCH STATUS 99 

4.2 MONTHLY AVERAGE HOURS OF EDUCATION 

OR TRAINING RECEIVED, BY RESEARCH STATUS 102 

4.3 MONTHLY RECEIPT OF BASIC EDUCATION, 

BY RESEARCH STATUS 113 

4.4 MONTHLY RECEIPT OF TRAINING, 

BY RESEARCH STATUS 115 

5.1 MONTHLY EMPLOYMENT RATES, BY GENDER 

AND PARENTAL STATUS 133 

5.2 MONTHLY RATES OF POSITIVE ACTIVITY 

FOR MEN, BY RESEARCH STATUS 136 

5.3 MONTHLY RATES OF POSITIVE ACTIVITi' 

FOR WOMEN LIVING WITH THEIR OWN CHILD(REN), 

BY RESEARCH STATUS 156 

5.4 MONTHLY RATES OF POSITIVE ACTIVITY 
FOR WOMEN NOT LIVING WITH THEIR OWN 

CHILD(REN), BY RESEARCH STATUS 157 



-xxxvii- 

ERIC 





ABBREVIATIONS 


AFDC 


Aid to Families with Dependent Children 


BSA 


Basic Skills Academy (New York, New York) 


CBO 


Community-Based Organization 


CCP 


Comprehensive Competencies Program 


GET 


Center for Employment Training (San Jose, California) 


CETA 


Comprehensive Employment and Training Act (1973) 


CREC 


Capitol Region Education Council (Hartford, Connecticut) 


EGOS 


Emily Griffith Opportunity School (Denver, Colorado) 


FSA 


Family Support Act (1988) 


GAO 


U.S. General Accounting Office 


GED 


General Educational Development Certificate 


JOBS 


Job Opportunities and Basic Skills Training Program 


JTPA 


Job Training Partnership Act (1982) 


MDRC 


Manpower Demonstration Research Corporation 


MFSP 


Minority Female Single Parent Demonstration 


MIS 


Management Information System 


PIC 


Private Industry Council 


SDA 


Service Delivery Area 


SSI 


Supplemental Security Income 


TABE 


Test of Adult Basic Education 


VICI 


Ventures in Community Improvement 


WIN 


Work Incentive Program 


YIEPP 


Youth Incentive Entitlement Pilot Projects 


YOU 


Youth Opportunities Unlimited Demonstration 



ERLC 



-xxxviii- 



CHAPTER 1 



AN OVERVIEW OF THE JOBSTART DEMONSTRATION 

Most young people make the transition from adolescence to employment and self- 
sufficiency between the ages of 16 and 24 However, many others - especially high school 
dropouts with poor skills - fail to do so, even in a period of strong economic growth such as 
the mid to late 1980s. Tie negative consequences extend well beyond the lives of the young 
people, affecting both the ^^eneral public and the business community. There is strong evidence 
that the incidence of poverty, welfare receipt, criminal activity, and unwed parenthood is 
significantly higher for those with poor basic skills than for the population as a whole. ^ Society 
bears the cost of this in the torm of social disruption and increased public services. 

Employers and the business community also suffer from young people's lack of job-related 
skills, and they are likely to find this an even greater problem in the future. The U.S. 
Department of Labor projects that in the year 2000 the number of young people in the labor 
force - a major source of entry-level workers - will be basically unchanged, even though the 
economy as a whole will have expanded considerably.^ Further, a growing proportion of these 
young people will come from groups with traditionally higher-than-average school dropout rates 
and basic skills deficiencies - minorities, recent immigrants, youths from single-parent families, 
and the poor. Basic skills shortages could translate into jobs going unfilled, employers having 
to pay higher wages to attract skilled workers, and some jobs needing restructuring to 
accommodate the available work force. 

What will help disadvantaged young school dropouts lead more productive lives? A 
number of experts have called for programs of education and training. The JOBSTART 
Demonstration tested such a program by combining basic education, training in occupational 
skills, limited support services (primarily assistance with child care and transportation), and job 
placement assistance. Developed and overseen by the Manpower Demonstration Research 
Corporation (MDRC), JOBSTART was implemented in 13 sites: four adult schools (three aduli 



^Berlin and Sum, 1988. 
^Fullerton, 1989. 

^William T. Grant Commission on Work, Family and Citizenship, 1988; Public/Private Ventures, 
1990; Job Training Partnership Act Advisory Panel, 1989. 
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vocational schools and one community college), six community-based organizations (CBOs), and 
three nonresidential Job Corps programs. The demonstration ran from 1985 to 1989, with 
operating support consisting primarily of funds provided under the Job Training Partnership 
Act (JTPA), the nation's largest funder of employment and training programs for economically 
disadvantaged persons.^ 

For an employment program like JOBSTART to improve the lives of young people, 
certain things must happen: 

• The sites must recruit young, economically disadvantaged school dropouts 
with poor skills (a group that is typically not served even in intensive 
programs). 

• The sites must put in place a package of services that addresses the needs 
of the youths. 

• The young people must respond favorably to this opportunity and make 
an investment of their time and effort by participating in education and 
training activities. 

• Their efforts must yield them new skills, as measured, for example, by 
their completing high school or passing the General Educational 
Development (GED) Test for high school equivalency certification.^ 

• Over time, these new skills must translate into greater employment and 
earnings than these youths otherwise would have had, and less need to 
rely on public assistance.^ 

Understanding whether these changes did occur is the goal of the JOBSTART evaluation. 

It is an open question whether helping young disadvantaged people increase their 
educational attainment will lead to increased earnings, especially in the short run. Numerous 
studies have found that people with higher levels of education earn more than do those 



"^Such funds came from Title IIA of JTPA, the largest portion of the JTPA program, or Title IVB, 
which funds the Job Corps Centers. 

^The General Educational Development Test is a national examination produced and administered 
by the GED Testing Service of the American Council on Education in Washington, D.C. States have 
different criteria for who may take the examination, different passing scores, and different credentials 
awarded to those passing (for example, a state high school equivalency certificate or a state high school 
diploma). In accordance with common usage, the credential is referred to in this report as a GED 
certificate or, simply, a GED. 

^le program could also affect other aspects of the young people's lives. They might be better able 
to live on their own instead of with their parents, more likely to postpone childbearing (because they see 
opportunities in the labor market), and less likely to engage in criminal or other antisocial behavior. 
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without a high school diploma. For example, one estimate, based on the experiences of young 
men and women during the late 1970s, found that the payoff of an additional year of 
secondary school was approximately $700 in increased annual income, while a high school 
diploma had a "credential effect" of about $925 per year J But direct rigorous tests of the 
impact of increasing the educational attainment of a group of disadvantaged young people who 
did not complete high school are rare. 

Although, in general, increasing the skills of young people does increase their earning 
capacity, there are two important countervailing effects of a program like JOBSTART, at least 
in the short run. Participation in an intensive program "pulls" people out of the labor force. 
Not only do they give up earnings while they participate in the program, but they also have 
less chance to gain skills and seniority through their on-the-job experience. For young people, 
especially those with low levels of educational attainment, work experience is an important 
source of new skills and increased job stabihW and wages. 

Further, little is known about how employers assess receipt of a GED, the primary 
educational outcome in "second chance" programs like JOBSTART. The assumption behind 
such programs is that employers will view a GED as evidence of increased skills, but virtually 
nothing is known about how long it takes for a GED to pay off and how this might vary 
among subgroups of youths and types of occupations. Alternatively, and less optimistically, a 
GED may do little to counter the negative impression created by the fact that the young 
person did not finish high school.^ 

Earlier reports on the JOBSTART Demonstration, summarized and updated in this 
document, found that the first four conditions for program impacts listed above were generally 
met: The programs recruited the target group of youths; they generally offered the intended 



^Berlin and Sum, 1988. These estiirates attempted to control for the level of basic skills of 
individuals, by including youths* scores on iiie Armed Forces Qualifying Test, administered as part of a 
special survey of youths, as an independent variable in a regression. The estimated effect on annual 
earnings of an additional grade-equivalent of basic skills (for example, progressing from a seventh- to an 
eighth-grade reading level) was $185. 

®An unpublished analysis by Andrew Sum, using the Current Population Survey, finds that the average 
earnings of high school dropouts increase with age noticeably during the late teenage years and early and 
mid 20s. In 1986, for example, the annual average earnings of 19-ycar-olds were about $2,000 higher than 
ihu^ for 18-year-olds. 

' disproportionately higher labor market reward for completing high school than for dropping out 
part-way through may indicate that employers rely on high schools more for screening workers than for 
teaching them basic skills (see Layard and Psacharopoulos, 1974). If this is true, attaining a GED after 
dropping out may add little to a young person*s employability. 



services; the youths participated in education and training; and this led to increased GED 
receipt. Thus, the JOBSTART Demonstration presents a unique opportunity to see whether 
and how enhanced educational attainment and vocational skills training for disadvantaged 
youths are "processed" by employers in the labor market. 

This report, covering two years in the lives of the young people in the demonstration, 
is about the JOBSTART program's early effects. As with most investments, JOBSTART had 
up-front costs. For the young people, the costs were (1) the time and effort devoted to 
participating in the program, (2) the income forgone by not working, and the lost 
opportunity to learn new skills on the job. For the public, the cost was financial support for 
the program activities. As with many other investments, the costs were incurred quickly, while 
the benefits may accrue over an extended period. 

With only two years of follow-up on the experiences of the young people, this is an 
interim report rather than the final story on JOBSTART. It summarizes the project's policy 
goals and research design; describes the nature and intensity of the education, training, and 
other services the young people participated in; and presents 24-month program impacts on 
educational attainment, employment, earnings, receipt of public assistance, and other outcomes. 
Throughout, the analysis examines whether the program has worked differently for subgroups 
of the young population it served. 

This chapter discusses aspects of ihe youth employment problem that shaped the program 
model and its evaluation. It then examines the policy and programmatic context in which 
JOBSTART emerged, highlighting how the rules and reality of the JTPA system strongly 
affected the demonstration guidelines. It also describes how events have made the findings 
relevant to a larger policy community seeking ways to serve young dropouts, including young 
mothers receiving public assistance. The chapter ends by summarizing the service guidelines 
for the program, important differences among the 13 sites, and the topics addressed in the 
chapters that follow. 



^°Sce Auspos, 1987; Auspos et al., 1989. 

"Understanding more about the longer-term labor market value of a GED is a central goal of the 
four-year follow-up to be analyzed in the final report on the JOBSTART Demonstration. That report 
will examine longer-term program impacts. 
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L The Youth Employment Problem and the Challepge of Serving Young School 
Dropouts 

At the heart of the youth employment problem is "a small group of young people who 
remain out of work a large portion of the time."^^ Overwhelmingly, they are from poor 
families, have dropped out of school, and lack the basic skills needed to succeed in the job 
market. Many are members of minority groups, some of whom confront the continuing 
existence of job discrimination. Young people who have dropped out of school and are 
unemployed - or only sporadically employed - may face lingering negative consequences. 

A, A Closer Look at the Employment Problems of Young School Dropouts 

In an economy increasingly reliant on service sector employment, it is virtually impossible 
for young school dropouts to find stable, well-paying jobs. Numerous national studies have 
found that dropouts are more likely than other young people to be poor academic achievers 
who had trouble fitting into a structurr'd environment (the schools) and who lack self-esteem 
and - often - the habit of persevering in complex and challenging tasks. Employers value 
the very skills and qualities dropouts often lack: They want employees who have the basic 
academic skills and a readiness to learn and to set goals, communication skills, self-esteem, and 
the motivation to follow through. Many black and Hispanic youths face a further obstacle 
to employment. They are more likely than whites to drop out, but less likely to have access 
to informal networks that are useful for finding well-paying, steady work. 

The resulting employment statistics highlight graphically the circumstances of young 
school dropouts. Those young people without four years of high school are much less likely 
to be working and more likely to be out of the labor force or (if in the labor force) 
unemployed.^^ For example, as Table 1.1 shows, in 1990, 76 percent of all 16- to 24-year- 



^^Clark and Summers, 1982, p. 200. See also Ellwood, 1982; Rees, 1986; Ilahn and Lerman, 1985, 
p. 6. Using data from the late 1970s, the Congressional Budget Office estimated that about 10 percent 
of all youths accounted for 61 percent of all youth unemployment (U.S. Congress, 1982, p. 12). 

^^ationally, about 15 percent of all people aged 20 to 24 have not completed four years of high 
school. See U.S. Department of Education, 1988. 

^'^e research on which this section is based is summarized in Ekstrom et al., 1986. 

^'American Society for Training and Development, 1988. 

^^e labor force is defined in official statistics as individuals who are employed or actively seekinj; 
employment. Unemployment statistics exclude people vho are not in the labor force. Thus, many people 
who are not working are not counted as "unemployed'' because they are not actively seeking work. Figures 
in this section are from U.S. Department of Labor, 1991. 



TABLE 1.1 



LABOR FORCE STATUS AND EMPLOYMENT RATES FOR 
16- TO 24-YEAR-OLDS NOT ENROLLED IN SCHOOL IN 1990 



Subgroup 


Percent 
Empl oyed 


Percent in 
Labor Force 
and Unemployed 


Percent Not in 
Labor Force 


All 16- to 24-year-olds 


71.2 


9.0 


19.8 


High school graduates with 
some subsequent education 


85.6 


5.6 


8.8 


High school graduates with 
no subsequent education 


75.6 


8.7 


15.7 


School dropouts* 


50.6 


12,6 


36.8 


White school dropouts 


55.3 


11.3 


33.4 


Black school dropouts 


29.0 


19.7 


51.3 


Hispanic school dropouts'' 


57.1 


9.3 


33.6 



SOURCE: U.S. Department of Labor, 1991. 



NOTES: The labor force is defined as individuals who are employed or 
actively seeking employment. 

Rows may not total 100.0 percent because of rounding. 

*Those who did not complete four years of high school but received 
a GEO are defined as school dropouts. 

'^Persons of Hispanic origin may be of any race and thus are 
includei in both the white and black school dropout subgroups. 



olds with four years of high school and do subsequent schooling were working, while 16 
percent were not in the labor force, and 9 percent were in the labor force but were 
unemployed. Among 16- to 24-year-olds without four years of high school, only 51 percent 
were working; 37 percent were outside the labor force; and 13 percent were in the labor force 
and unemployed. In this fiame age group, only 29 percent of blacks were working, while 51 
percent were outside the labor force, and 20 percent were in the labor force and unemployed. 
Among the same-aged whites and Hispanics, about 55 to 57 percent were working, while 33 
to 34 percent were not in the labor force, and about 9 to 11 percent were in the labor force 
and unemployed. 

B. The DifTerent Circumstances of Young Women and Young Men 

In many obviom and important ways, the lives of young men and young women differ. 
This carries through to the experiences of young people who drop out of school. For young 
men, the last two decades have brought a deterioration in the ability to earn a living, with the 
decline in earnings being most serious for those with poor skills. For young women, single 
parenthood - and scant financial resources for handling it — is a common predicament. 

During the 1970s and 1980s, young men without a high school diploma experienced a 
sharp decline in average earnings (after adjustments for inflation).^® For example, between 
1973 and 1986, the average real earnings of 20- to 24-year-old men who had not graduated 
from high school and were not enrolled in school declined 42 percent. The picture for black 
men in this group was especially bleak, with their average real earnings dropping 61 percent. 
During the same period (which included serious recessions), the earnings of male high school 
giaduates declined 28 percent, while the earnings of male college graduates declined only 6 
percent. 

The incidence of unwed teenage parenthood increased during the last two decades. 
While the number of births to American teenagers actually declined between 1970 and the late 
1980s, the proportion of those births that were outside marriage rose dramatically, from about 



^^This employment-to-population ratio is calculated by dividing all those who were employed in 1990 

by the total civilian noninstitutional population. 

^^William T. Grant Commission on Work, Family and Citizenship, 1988, pp. 26-27, 

^^is bleak picture for black male school dropouts also appears in labor force statistics of the type 

cited earlier. Unpublished U.S. Department of Labor data for 1989 show that only 42 percent of 18- to 

24-year-old black men without a high school diploma and not enrolled in school were employed, compared 

to 69 percent of comparable whites and 75 percent of comparable Hispanics. 
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30 percent (200,000) in 1970 to about 65 percent (300,000) in the late 1980s. Giving birth as 
a teenager typically coincides with a disruption in a mother's education and greatly increases 
her labor market difficulties,^ Unmarried women who become mothers in their teens and do 
not complete high school are at great risk of long-teuw welfare dependence and 
unemployment.^^ In addition, recent evidence indicates that while teenage mothers can often 
recover partially from the detrimental effects of early childbearing, their children are more 
likely to suffer long-term social and educational disadvantages than are children born to older 
women.^ 

C* Research on Effective Programs for Young School Dropouts 

At the time the JOBSTART Demonstration began, program designers seeking insights 
from the previous research on youth programs found few solid success stories on which to base 
new efforts. Many programs had been tried, but nearly all evaluations found unfavorable 
results, were inconclusive, or were seriously flawed.^ 

The one influential exception to this pattern was the residential Job Corps, which a 
study found to be effective in increasing the educational attainment and earnings of young 
dropouts.^ The residential Job Corps provides basic skills education, occupational training, life 
skills instruction, work experience, job placement assistance, health care, counseling, and other 
support services to youths who live at centers (often outside urban areas) and participate in 



^In a national survey of high school students, almost one-fourth of all young women who dropped 
out of school reported that pregnancy was an important factor in their decision. See Ekstrom et aL, 
1986, for a discussion of this study. 

^^See Ellwood, 1986. In 1989, only 42 percent of 18- to 24-year-old white women without four years 
of high school and not currently enrolled in school were employed; for blacks, the comparable figure was 
22 percent and for Hispanics, 32 percent The percentages for young school dropouts who were also 
mothers would be even lower, but were not calculated owing to sample size limitations in the Current 
Population Survey. 

^^ee Furstenberg, Broolcs-Gunn, and Morgan, 1987. 

common methodological problem was the absence of an appropriate group (one that was similar 
to participants but not served by the program) against which the experiences of the group that was served 
could be compared. Without such a comparison, evaluators frequently confused outcomes that followed 
a program with the real difference a program made - in the language of evaluation, its "impacts/ For 
example, the outcomes of a program might include a post-program job placement rate of 50 percent. 
However, the employment rate for the appropriate comparison group might also be 50 percent, suggesting 
that the program had no impact on employment rates. See Betsey et al., 1985, and the discussion in 
Chapter 2 of this report for more on this issue. 

^'^See Mallar et aL, 1982. 
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the program for up to two years. About 80 percent of Job Corps participants have not 
completed high school. The residential Job Corps, however, could not be offered to all 
dropouts: It was a relatively expensive program, accessible only to those willing and able to 
live away from home, requiring devel'' )ment of work experience positions with employers, and 
clearly not the answer for all disadvantaged youths. 

Other efforts to directly connect young people with work - by either helping them look 
for work more effectively or providing subsidized work experience - were tested in 
demonstrations in the early 1980s. The evaluation of job search assistance for youths found 
that the program produced short-term increases in employment and earnings, but that in the 
long run participants were no better off than a comparison group.^^ 

As for the most common youth employment strategy - subsidized work experience - 
two evaluations failed to find any long-term impacts on educational attainment, employment, 
or earnings for dropouts. The National Supported Work Demonstration, managed by MDRC 
in the late 1970s, enrolled very disadvantaged young dropouts (many with a criminal record) 
in a 12- to 18-month program of paid work experience with gradually increasing job 
responsibilities. Program impacts for this group were not positive, even though the program 
proved successful for long-term welfare recipients.^^ The Youth Incentive Entitlement Pilot 
Projects (YIEPP), which offered subsidized minimum-wage jobs to high school students and 
dropouts who returned to school, also was ineffective for dropouts.^ While the program did 
increase the employment and earnings of young people still in school, evaluators found thai 
the offer did not induce dropouts to return to and remain in regular high school. Many of 
those who did return dropped out a second time, and there were no effects on educational 
attainment, employment, or earnings for dropouts. 



^Some Job Corps Centers also operate a nonresidential program. Tliese were not included in this 
earlier study. As previously noted, three nonresidential Job Corps programs were included in the 
JOBSTART Demonstration. 

^^e demonstration assessed the effectiveness of a program providing job search assistance through 
simulated interviews, seminars on job-seeking techniques, and help in making contact with potential 
employers. See Public/Private Ventures, 1983. This finding differs from that of research on job search 
assistance programs for ^omen receiving Aid to Families with Dependent Children (AFDC), which did 
find long-term employment and earnings impacts. See Gueron and Pauly, 1991. 

2^See Maynard, 1980, on the findings for young dropouts and Manpower Demonstration Research 
Corporation, 1980, for the results for AFDC recipients. 

^See Gueron, 1984. 



Thus, the research record of the mid 1980s put the Job Corps in a special category as 
an effective program for raising the employment and earnings of young school dropouts.^^ As 
would be expected in an intensive program of skills enhancement, Job Corps participants 
initially earned less than their comparison group counterparts.^^ This pattern of earnings 
continued for a time after people left the program because those in the comparison group had 
accrued more work experience (an important source of skills-building), seniority and protection 
against layoffs, and promotions.^^ The positive effects of enhanced human capital did not 
outweigh the negative effects of lost work experience until roughly 6 to 12 months after 
participation in the program, the first semi-annual period in which the average earnings of 
program participants exceeded those of their comparison group counterparts. 

Not only did the research record find positive impacts for the residential Job Corps 
program, but it also found program benefits to be greater than program costs. Although 
program costs per participant were much higher than for most other programs (averaging 
about $5,000 in 1977), the estimated benefits to society exceeded the costs. Especially 
encouraging was the program's positive findings for young male dropouts, a group that had 
proven especially hard to serve in many previous programs. 

Among the questions left open by the existing research, however, was whether the Job 
Corps approach could be successfully adapted to a new setting: operation as a nonresidential 
program by other agencies, which could not offer comprehensive support services and probably 
would not have the Job Corps' level of resources to devote to staff training, facilities, and 
curriculum. This shift to a nonresidential program is an important one, since some of the 
benefits of the traditional Job Corps program seem to stem from its residential nature. For 
example, in a residential program, it is much easier to provide an intensive program of support 
services (including counseling outside class time, positive peer support, recreational activities, 
and health care) than when young people are active in the program for at most eight hours 
a day. Furthermore, the decline in criminal activity and substance abuse observed for Job 



National Academy of Sciences, in its review of research on employment programs for young 
people, pointed out the distinction between the failure of research to provide adequate evidence of 
program effectiveness and the finding that a program is ineffective. Betsey et al., 1985. 

^^is would occur because of participants' forgone earnings and lost opportunities for on-the-job 
skills enhancement while they were in the program, as discussed earlier in this chapter. 

^^Evaluations of the Job Corps discussed the problem of the post-program transition back into the 
labor market that led to these initi^il negative impacts. Se<^ Mallar et al., 1978, 1980. 
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Corps participants (especially during program participation) was partly attributable to their 
isolation in residential centers outside urban areas, or at least outside their previous 
neighborhood. While the residential nature of the program may have been a factor in its 
success, it did pose problems for some young people. For young mothers with child care 
responsibilities, the program demanded too much time away from home, and it did not prove 
effective for them. Also, many young men and women did not wish to leave their 
communities. 

In 1983, the National Academy of Sciences convened a panel of experts on youth 
programs. Their assessment - summing up research findings - recommended further testing 
of the Job Corps approach in a nonresidential setting using random assignment to produce the 
most reliable firdings.^^ The JOBSTART Demonstration was, in part, a response to this 
call.3^ 

D. GroHlne Operational Ey p erience in Progrhms tor Young Dropouts 

Program operators ser/ing young, disadvantaged dropouts have identified a number of 
lessons that also informed the development of the JOBSTART Demonstration.^ When 
serving these youths, who often look back on past educational experiences with dissatisfaction, 
programs have to actively seek out participants rather than passively wait for volunteers to 
come forward. Program operators have also learned that achieving continued participation is 
not easy: Counseling and peer support have often proved useful in improving young people's 
self-esteem and motivation, but even with these efforts, participation levels can be 
disappointing. 

Finally, program operators have increasingly become sensitive to the multiple needs of 
clients. For some economically disadvantaged young people, a low level of basic skills prevents 
them from taking advantage of occupational training. In addition, as is the case for many 



^^Random assignment is generally recognized to be a reliable method of measuring the effectiveness 
of new programs of employment and training. As discussed later in this report, it was used in the 
JOBSTART Demonstration. 

•''^As the later discussion of the JOBSTART program model and its implementation will make clear, 
JOBSTART did not offer the same comprehensive list of support services available in the Job Corps. 
Nor did it use the same curriculum in education or training, except in the three sites that aheady operated 
a Job Corps nonresidential program. Nationally, about 10 percent of Job Corps participants are in 
nonresidential programs. 

^M.,ny of these lessons are summarized in 70001 Training and Employment Institute, 1988, and 
Public/Private Ventures, 1990. 
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young people, economically disadvantaged youths may not be experienced in setting goals, 
making plans to achieve them, and following through with effective action. And finally, many 
young people have a pressing need for immediate income, for their^selves or their families, so 
programs must help them find a means of financial support while they invest in their future 
by enhancing their skills. 

IL The Changine Programmatic Context of the JOBSTART Demonstration 

Because the JOBSTART Demonstration did not have special program funding to support 
site operations, it was shaped in important ways by the need to find a funder for local 
JOBSTART sites within existing programs. Since JTPA was the most likely source of local 
operational funds, its provisions and the local interpretation of them were central to the 
structure of the project. As the demonstration unfolded, JTPA changed in ways that make the 
JOBSTART findings even more relevant to the JTPA system. And a new audience for the 
demonstration findings has emerged with the passage of the Family Support Act of 1988, 
calling for expanded programs of education and employment services for young women who 
are school dropouts and receive welfare. 

A. JTPA as a Crucial Funder of Program Services 

The Job Training Partnership Act (JTPA) of 1982 is the federal government's major 
program for funding employment and training for economically disadvantaged adults and 
youths. JTPA distributes the majority of its funds to states which, in turn, pass along most of 
what they get to local administrative entities called service delivery areas (SDAs).^5 The 
federal JTPA statute sets general rules for program eligibility and allowable types of activities. 
An SDA's staff and private industry council (PIC) - often operating like a board of directors 
for the agency - determine what specific types of services are to be offered, which groups will 
get priority for services, and how service providers under contract to the SDA are to be 
evaluated and paid. 

The manner in which JTPA was initially implemented during the mid 1980s presented 
operational constraints that had to be taken account of in the design and implementation of 
the JOBSTART Demonstration: 



^^Most JTPA funds under Title IIA, the largest part of JTPA, are distributed to states, using a 
formuia based on the states^ number of unemployed and eco]$omically disadvantaged people. 
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• Performance standards that made SDAs hesitant to serve youths with 
very poor skills. The incentives embedded in Title IIA, the largest part 
of JTPA and the one that finances most youth programs, made SDAs and 
JTPA-funded education and training agencies hesitant to enroll youths with 
very poor basic skills who were in need of intensive programs of education 
and training and support services. In designing and applying the 
performance measures used during the first five years of JTPA, federal, 
state, and local administrators focused on the proportion of participants 
placed in a job, their wages, and the cost per "success story. This 
encouraged SDAs and service providers to choose people who were more 
likely to achieve these successes at relatively modest costs.^^ In seeking 
to serve school dropouts with poor skills in an intensive program, 
JOBSTART had to confront these incentives. 

• Performance contracts that inhibited combined education and training. 

Many SDAs wrote contracts with senice providers that linked payment 
to the achievement of the events measured in performance standards. 
This made it difficult for those service providers that wished to serve 
youths with poor skills to be paid and complicated the administration of 
program models that required both education and training when a single 
agency could not provide both. 

• Severe restrictions on paid work experience. Experience with public 
service employment under JTPA's predecessor, the Comprehensive 
Employment and Training Act (CETA), led Congress to eliminate most 
forms of paid work experience when enacting JTPA 

• Tight limitations on support services. The statute limited spending on 
support services (such as transportation and child care assistance) and 



^From the early 1980s until program year 1987 (ending in June 1988), the performance of SDAs 
serving adults was judged by the following standards: the percentage oi adults who found a job; the 
percentage of adults who were receiving welfare when they enrolled in JITA and who found a job; the 
average wage at placement in a job; and the program cost per person entering employment. For youths, 
the standards included the percentage who found employment and the "positive termination rate," defined 
as entering employment or other quantifiable measures of program success. These included attainment 
of employment competencies recognized by local private industry councils, completion of a level of 
schooling, enrollment in further non-Title IIA training, enlistment in the armed forces, return to school 
full time, or (for 14- and 15-year-olds) completion of specified program objectives. The youth standards 
included the cost per "positive termination." For each measure, the U.S. Department of Labor set national 
levels, which - at state option - could be adjusted to reflect the characteris-.i.\i of those served and the 
conditions in the local labor market. 

3'^Data from the mid 1980s illustrate the effects of these program priorities. During program ye;»rs 
1984 to 1986, when the JOBSTART Demonstration was beginning, young dropouts constituted only w 
percent of all Title IIA participants and 27 percent of all young participants. Among young dropouts 
who were served under Title IIA nationally in 1986, only 23 percent received basic education, a service 
likely to promote their long-term employability but unlikely to lead to immediate placement in a job. 
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needs-based cash payments and completely eliminated the payment of 
stipends to partidpants. 

The early experience under JTPA prompted allegations that the program was making 
little real difference in participants' lives because service providers tended to enroll more job- 
ready applicants (a practice known as "creaming").^ Further, sharp declines in the 
unemployment rate during the 1980s, which allowed many more job-ready individuals to find 
work, caused a rethinking about whether JTPA should continue to emphasize quick placement 
of participants in a job. Over time. Congress, the U.S. Department of Labor, and program 
operators have all expressed renewed interest in intensive programs of education and training 
targeted on more disadvantaged youths. 

Responding to the early pattern of program operation, the U.S. Department of Labor 
changed itS administrative practices and regulations and encouraged greater provision of 
intensive services for youths through a formal demonstration of such programs. In late 1987, 
the Department stated that "more emphasis must be placed on intensive investments in youth 
within JTPA" and recommended that "a significant portion of youths who participate . . . 
should receive competency-based instruction in either basic education or occupational skills."^^ 
Soon thereafter, amendments to the regulations (effective in program year 1988) encouraged 
states to choose as the key standard for youth programs one that includes measures of 
increased educational and skills competencies. This increased the opportunities to include 
young dropouts with poor skills in JTPA In addition, the Department in 1989 issued a 
request for proposals^ for the Youth Opportunities Unlimited (YOU) Demonstration and in 
1990 selected seven sites to operate innovative programs.*^ 



U.S. Department of Labor has contracted for an independent study of JTPA to determine 
program impacts in a sample of 16 SDAs across the country. The stu^y is being conducted by Abt 
Associates Inc, MDRC, NORC, and Lewin-ICF. See Doolittle and Traeger, 1990. The first impact 
results will be available in early 1992. 
^^Federal Register, December 16, 1987. 

^^Sites applying to participate were required to operate one of three programs; a work experience 
program modeled on Ventures in Community Improvement (VICI), which operated from 1978 to 1980; 
an alternative high school program modeled on High School Redirection in Brooklyn, New York; and a 
program of education and training modeled after JOBSTART. Seven sites were chosen for the three- 
year demonstration; some of them chose to operate a program modeled after JOBSTART. An 
implementation study is part of that demonstration, which is separate from the JOBSTART Deu:onr*ration 
reported on here. 
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Debate over further changes in JITA also signaled greater interest in more intensive 
programs for young dropouts. An advisory committee to the U.S. Department of Labor 
recommended shifting more JTPA resources to harder-to-serve youths and endir g restrictions 
on the support services these youths are likely to need.'*^ Although JTPA was not amended 
in the 1989-90 session of Congress, nearly all of the proposals would have abolished or 
downplayed cost standards for youth programs and required greater targeting of program 
resources on youths with multiple barriers to employment, a group likely to need more 
intensive services. 

While this interest in hard-to-serve youths came too late to affect the implementation 
of the demonstration, it has heightened the importance of the project as an early test of a new 
direction for JTPA and has increased the chances that the JOBSTART program will be 
successfully replicated if the research findings are positive. 

B. The Family Support Act 

Passage of the Family Support Act of 1988 (FSA) signaled a growing emphasis on 
programs of education and employment services for AFDC recipients, making the JOBSTART 
findings relevant for organizations serving this group. The Job Opportunities and Basic Skills 
Training (JOBS) title of the Act expands the obligations of AFDC mothers to participate in 
activities intended to increase their employability, and of states and counties administering the 
AFDC program to offer more education and training than were typically offered under the 
predecessor Work Incentive (WIN) program. Especially relevant to JOBSTART is the fact 
that the JOBS legislation allows states to impose a participation obligation on AFDC parents 
under 20 years of age who lack a high school diploma or GED regardless of the age of their 
child.'*^ For this group, education is normally presumed to be the appropriate first activity. 
States and local service providers seeking to expand their offerings of education and training 
for young mothers are currently grappling with many of the issues addressed in JOBSTART. 




"^Job Training Partnership Act Advisory Panel, 1989. 

''^While single parents with children under age three are normally exempt from participaiion in JOBS 
programs, this is not the case for custodial parents under age 20 who have not graduated from high school 
or received a GED. 
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III. Th e JOBSTART Demonstration 

MDRC began the JOBSTART Demonstration in 1985 with two purposes: (1) to 
determine the operational feasibility within JTPA of an intensive program incorporating several 
of the key elements of the residential Job Corps, and (2) to rigorously test its effectiveness.'*^ 
Local and state JTPA agencies provided most of the operational funding for the JOBSTART 
sites, but the MDRC evaluation was funded by an unusual consortium consisting of the U.S. 
Department of Labor, the Rockefeller Foundation, the Ford Foundation, Charles Stewart Mott 
Foundation, the William and Flora Hewlett Foundation, the National Commission for 
Employment Policy, AT&T Foundation, Exxon Corporation, ARCO Foundation, /Etna 
Foundation, the Chase Manhattan Bank, and Stuart Foundations. Funding trom this 
consortium also enabled MDRC to award a modest $25,000 grant to each site. 

This funding structure shaped the character of the demonstration at the local level in 
two important ways. First, the JOBSTART program operated within existing agencies and 
programs under the rules and performance standards of Title IIA of JTPA or, for the 
nonresidential Job Corps Centers, under Title IVB of JTPA It proved a serious challenge for 
the non-Job Corps sites simultaneously to follow the demonstration guidelines, the rules of 
Title HA of JTPA, and the provisions in their contracts with SDAs. 

Second, without special funding, sites could not be expected to make n\ajor changes in 
their existing programs, limiting the extent to which the JOBSTART curriculum and 
instructional methods could be standardized. Instead, MDRC gave sites general guidelines for 
program operation specifying the type and duration of required components of the program 
(education, occupational training, job placement, and support services). Even within this 
flexible framework, some program operators faced major implementation challenges. Some of 
the sites normally offered only basic skills education or vocational training; the demonstration 
called for both, requiring them either to add a whole new kind of activity or to link up with 
other local agencies providing it. Some sites also had to adapt to a younger and less skilled 
student I ody than they normally served. The lack of special program funding also limited the 
extent to which non-Job Corps sites could offer the array of support services that were a part 
of the Job Corps program. 



^^See Auspos, 1987, for a discussion of the origins of JOBSTART and its early implementation. 
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The demonstration was thus a hybrid: part evaluation of existing programs and part test 
of a new program. The basic program differed from site to site in myriad details, but the 
variety did permit a test of how a scaled-down Job Corps-type program could operate under 
existing rules in different kinds of established agencies. If the demonstration showed positive 
results, it would be easier to replicate the program widely. 

A. The .TOBSTART Sites 

MDRC staff recruited sites each of which they thought could (1) meet the JOBSTART 
program guidelines (discussed below) with little or no technical assistance except on techniques 
of client outreach and retention, (2) assemble sufficient operational funding for the full array 
of JOBSTART services (a significant barrier, as discussed above), and (3) yield a target of 200 
sample members. A total of 13 sites across the country, listed in Table 1.2, participated in the 
demonstration.*^ All had experience running programs that included some or all of the 
components of the JOBSTART model or working with young dropouts. As noted earlier, most 
of the operating funds for the demonstration sites were provided through the regular JTPA 
system under Title HA of the legislation. (The Job Corps, as also noted earlier, is separately 
funded and administered under Title IVB of JTPA, so the three Job Corps Centers in 
JOBSTART received funding through that title.) 

While all agreed to implement the JOBSTART model, the sites brought to the 
demonstration varying operating experiences: 

• Sponsoring organizations. The participating organizations included adult 
vocational schools, a community college, community-based organizations 
that focus on literacy development and GED preparation, community-based 
organizations that focus on occupational skills training, and the 
nonresidential components of three Job Corps Centers.^^ 

• Prior service emphasis. Some sites previously had offered only basic 
education and no skills training, while others had offered both but had 
emphasized skills training. The education-focused sites may have attracted 
youths who were primarily interested in basic education rather than skills 
training. Similarly, some sites with strong histories of skills training may 
have attracted youths who were primarily interested in learning the skills 
needed for a particular occupation rather than attaining a OLD. 



'♦^See Auspos, 1987, and Auspos et al., 1989, for a detailed discussion of the characteristics of the sites 
in the demonstration. 

''^The Job Corps Certers operated their usual nonresidential programs. Thus, they offered all 
JOBSTART services plus other Job Corps services that are not part of the JOBSTART model. 
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THE JOBSTART SITES 



Agency Name 



Location 



Type of Organization Prior Service Emphasis* JOBSTART Program Structure*^ 



00 



5 S 
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Allentown Youth Services 
Consortium^ 

Atlanta Job Corps 

Basic Skills Academy (BSA) 

Capitol Region Education 
Council (CREC) 

Center for Employment 
Training (CET) 

Chicago Commons Association's 
Industrial and Business 
Training Programs 

Connelley Skill Learning 
Center 

East Los Anjeles Skills 
Center 

El Centro Communilty College 
Job Training Center^ 

Emily Griffith Opportunity 
School (EGOS) 

Los Angeles Job Corps 

Phoenix Job Corps 

SER/Jobs for Progress 



Buffalo. NY 

Atlanta, GA 
New York, NY 
Hartford, CT 

Sar Jose, CA 

Chicago, IL 

Pittsburgh, PA 



Monterey Park, 
CA 

Dallas, TX 



Denver, CO 

Los Angeles, CA 

Phoenix I AZ 

Corpus Christi , 
TX 



Community-based 

Job Corps Center 
Community-based 
Cofrmunity- based 

Community-based 

Community-based 



Arult vocational 
"/Chool 

Adult vocational 
school 



Adult vocational 
school 

Job Corps Center 
Job Corps Center 
Community-based 



Education 

Education and training 

Education 

Education 



Training with some 
education 

Training 



Education and training 
Education and training 
Training 



Sequential /brokered 

Concurrent 
Sequenti al /brokered 
Sequential /brokered 

Concurrent 

Concurrent 



Education and training Concurrent 



Education and training Concurrent 



Comnunity college Education and training Sequential/in-house 



Education and training Concurrent 



Sequential/in-house 



Concurrent 



Concurrent 



5'J 



(continued) 



TABLE 1.2 (continued) 



NOTES: ^'*Education" refers to basic education, often as preparation for the GEO examination, "Training" refers to 
instruction in occupational skills needed for specific jobs. 

^Concurrent programs offer basic education and occupational training concurrently from the beginning of participation. 
Sequential /in-house programs offer basic education followed by occupational training, with both components provided in-house by the 
agency. Sequential /brokered programs provide basic education and then serve as a broker for occupational training, referring 
participants to other agencies. 

^In October 1990 this site was renamed The Clarkson Center, Inc. 

^In September 1988 this site was renamed the Edmund J. Kahn Job Training Center. 



This diverse background led the sites to implement the basic JOBSTART program 
components in several ways. Eight sites were able to offer both education and training in- 
house and chose to provide them concurrenlly, with participants active in both activities from 
the start. Two sites provided both activities i i-house, but offered them in sequence, with skills 
training following educatioih The remaining three sites did not have the capacity to offer skills 
training and chose to provide basic education themselves and work with other agencies to place 
their participants in subsequent occupational training elsewhere. 

B. Thfc JOBSTART Program Guidelines 

Drawing on the lessons of the Job Corps and applying them within the constraints of 
JTPA, the demonstration developed a new alternative program offered in a nonresidential 
setting with fewer support services available to participants. The key elements, shown in Table 
13, included the core components of the Job Corps (basic education, occupational training, 
and job search) but a less extensive system of support services-"*^ In some respects (the 
defmition of the target population and the requirement that certain activities be included), the 
program model was quite specific, while in others it allowed for considerable variation. The 
model set requirements as to the type and intensity of education and training services that 
were to be offered to participants, and it placed strong emphasis on the need for strategies to 
increase program retention. However, as mentioned earlier, sites were given a great deal of 
flexibility in implementing these core requirements. 

1. Target group. Since the program was designed to reach a population largely 
unserved by existing programs, eligibility requirements were quite specific. Participation was 
limited to school dropouts who were between 17 and 21 years of age, did not have a high 
school diploma or GED, read below the eighth-grade level, and satisfied the JTPA definition 
of economically disadvantaged (defined primarily by household income or receipt of public 
assistance). Recognizing that program operators needed to meet enrollment and performance 
standard targets, however, the guidelines allowed for up to 20 per ent of participants to read 
at or above the eighth-giade level 



"^Chapter 3 of this report provides more detail on the JOBSTART program model as implemented 
by the sites in the demonstration. A fuller description is given in Auspos et ah, 1989. 
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TABLE 1.3 
THE JOBSTART PROGRAM GUIDELINES 



Target Population To be eligible for JOBSTART, individuals had to be: 

17 to 21 years old 

school dropouts without a high school diploma or 
GEO 

reading below the eighth-grade level on a 
standardized test^ 
economically disadvantaged'^ 



Basic Education Sites were to implement a curriculum that: 

was self-paced and competency-based 
• was computer-managed and -assisted, if possible 
was a minimum of 200 hours in length 
focused on reading, communication, and basic 
computation skills 



Occupational Skills Training Sites were to implement a curriculum that: 

was in a classroom setting 
combined theory and hands-on experience 
prepared enrol lees for jobs in high-demand 
occupations 
• provided at least 500 hours of training 

had been developed with the assistance of the 
private sector to ensure that graduates would meet 
the entry-level requirements of local employers 



Training-Related Support Services Services were to be tailored to individual needs and 

were to include, in addition to transportation and 
child care, some combination of the following: 
work-readiness and life skills training 
personal and vocational counseling, mentoring, 
tutorial assistance, and referral to external 
support systems 

needs-based payments or incentive payments tied to 
length of stay, program attendance, or perfomance 



Job Development and Placement JOBSTART operators and/or their subcontractors were to 

Assistance be responsible for assisting participants in finding 

training-related jobs 



SOURCE: Manpower Demonstration Research Corporation, 1985. 

NOTES: *To help meet enrollment targets, each site was allowed to enroll individuals 
up to 20 percent of its total JOBSTART enrollment - who read at or above the eighth-grade 
level . 

'^To be eligible for JTPA services - economically disadvantaged by JOBSTART 
standards - a person must be receiving public assistance; have family income at or below the 
poverty line or 70 percent of the lowest living standard income level; be homeless, under 
the definition of federal statutes; or, in some cases, be a handicapped adult whose own 
income fits within the guidelines but whose family income exceeds it. 
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2. Education and training. The demonstration sought to test an intervention that 
would be relatively intensive and lengthy compared to the usual JTPA activities and that would 
address the multiple deficits in participants* skills. As a result, the program model required 
sites to offer a specified minimum amount of both basic education and occupational training 
to provide the young people with a real opportunity to enhance their skills. This combination 
of services, as noted earlier, differed from the usual situation under Title IIA of JTPA. The 
200-hour minimum of education was based on an estimate of what would be needed to bring 
the basic skills of most participants reading below the eighth-grade level up to the point where 
they could qualify for a GED or enter occupational skills training. The 500 hours of training 
was a compromise between the very lengthy training that research suggested was useful and 
what was practical in most JTPA environments. Given the difficulty of keeping young people 
engaged in a program for an extended period and the competing demands on their time 
(including their need for income and their child care responsibilities), staff recognized that not 
all participants would complete these activities and that the total time in the program would 
be a year or less. 

The two instructional components were structured in ways intended to make them 
appeahng and accessible to young people who entered the program wth widely varying levels 
of skills. For basic education, the guidelines required sites to ofier instruction in reading, 
communication, and basic computational skills, using individualized curricula that allowed youths 
to proceed at their own pace toward required competency goals. The program model did 
not specify any particular curricula, though it did encourage - but not require - sites to offer 
computer-assisted instruction. 

The occupational skills component required classroom rather than on-the-job training, 
in the belief that participants would benefit from the intensive, closely supervised instmction 
possible in a classroom setting. Again, no specific curriculum was required. Recognizing the 
advantages of applying learning to practical problems, however, the program model required 
that the training include a combination of theory and hands-on experience. Seeking to 
increase the chances of placement following training, the program model required that the 
training prepare participants for jobs in high-demand occupations and be developed in 
cooperation with local representatives of the private sector 



Sites were also expected to supplement this individualized instruction with group activities. 
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3. Support services. Attracting and keeping disadvantaged youths in education 
and training programs is a challenging problem, and the sites were expected to assist 
participants with transportation and child care. They were also encouraged to develop a 
package of other support services to facilitate program participation; the Job Corps sites 
offered considerably more support services than did the others.^ 

4. Job placement assistance. The guidelines required sites to identify possible 
training-related jobs for participants and to assist them in securing employment, but were not 
specific about how this should be done. All sites instructed the youths on work disciplines, 
employer expectations, and job search techniques, but the intensity of this effort ranged from 
informal guidance by counselors and other staff to more than 50 class hours in one site. Seven 
sites offered some form of work experience or internships (both paid and unpaid) to improve 
job skills. All sites provided assistance in seeking employment when the youths left the 
program, although in two of the three sequential/brokered sites (CREC in Hartford and BSA 
in New York City), the responsibility fell solely on the training provider. This arrangement for 
job search assistance proved a serious limitation since, as will be discussed in Chapter 3, many 
young people did not reach the training phase in sequential^rokered sites. 

C* Key Dimensions of Local Variation 

The previous report on the implementation of JOBSTART highlighted two dimensions 
of local variation as important influences on the program experience of the JOBSTART 
youths: 

* Concurrent versus sequential education and training. Programs could 
offer youths basic education classes and vocational skills instruction at the 
same time (a concurrent model) or basic education before skills training 
(a sequential model). 

• lu-house versus brokered services. Programs could offer youths education 
and training at the same agency^ or the agency providing basic education 
could serve as a broker, helping participants >vho were completing the 
education phase to fmd appropriate training at other institutions 
(sequential/brokered sites). 
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'**Job Corps Centers offered health services, reaeational activities, and on-site food service, and more 
intensive counseling and peer support than did most other sites. 
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In the period following the last JOBSTART report, research on other programs has also 
called attention to variations among sites offering concurrent programs. Hie Minority Female 
Single Parent (MFSP) Demonstration, funded by the Rockefeller Foundation, tested different 
models of education and training in four local agencies."^^ The early evaluation reports based 
on one year of follow-up argue that the one program among the four in the study with positive 
impacts on employment and earnings (CET, operating in the San Francisco Bay area) achieved 
this result because of programmatic and organizational features that distinguished it from the 
remaining three.^^ Specifically, researchers highlighted "the training design of the CET 
program - which emphasized training for all regardless of educational skill levels, offered 
remedial education within the context of job skill training, and accommodated trainees with 
diverse levels of educational skills."^^ The hallmark of such an integrated program is a focus 
on vocational training, aided by the teaching of literacy and mathematics skills needed in the 
chosen occupational area (rather than general hteracy and mathematics training). The U.S. 
armed forces have experimented with this approach in teaching military occupations to recruits 
with poor basic skills,^^ 

In practice, there is no clear distinction between integrated concurrent programs and 
other concurrent programs; they form a spectrum rather than falling into two neat categories. 
Among the JOBSTART sites, for example, CET/San Jose operated the most integrated 
program. Basic skills instruction was largely incorporated into the training curriculum, with the 
goal of supporting the learning of occupational skills. The challenge for this approach is to 
serve participants with very low levels of basic skills. Some people in this group will initially 
be unable to participate in serious vocational training, so program operators face a choice of 



''^See Gordon and Burghardt, 1990. 

CET site in the MFSP Demonstration enrolled minority female single parents, whose average 
age was 23, and served them in San Jose and several other East Bay communities. The San Jose program 
was also a site in JOBSTART, but only 10 mothers became part of the JOBSTART sample at CET/San 
Jose, and they were considerably younger than the MFSP sample. 

^^Gordon and Burghardt, 1990, p. xxvi. The authors also cited a number of other factors that were 
unique to CET among the four sites and that they believe contributed to its large impacts, including its 
financial stability and experienced staff; integration of the MFSP program into an ongoing training 
operation with a similar mission; large scale, which allowed for training in a variety of occupations in 
demand in the local economy; attention paid to job placement; and availability of on-site child care. The 
remaining three sites in the MFSP Demonstration emphasized "the acquisition of basic skills before entry 
into job skill training" (p. xxvi) - that is, sequential programs in the terminology used in the present 
report. 

^^is approach is sometimes call "functional context training." See Sticht, 1987. 
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offering them special help in literacy and mathematics or routing them to less demanding 
training courses. This trade-off is illustrated by the experiences of a second site, which 
operated a partially integrated program. Chicago Commons offered several training courses 
requiring technical knowledge and mathematics skills. Even after imposing entrance 
requirements among the most stringent in any JOBSTART site, Chicago Commons found that 
the skills deficiencies of some participants were too severe to be addressed within the 
integrated training context. Thus, the site also offered a separate basic education class. 
Other concurrent sites such as SER/Corpus Christi and Connelley in Pittsburgh operated 
separate education classes aimed at preparing people to pass the GED examination. These 
sites tried to coordinate the activities in education and training classes via conferences among 
the instructors and inclusion of basic skills instruction in some training classes. 

The strong policy interest in these issues has led to a lively debate about the arguments 
for and against each approach. 

1. Sequential versus concurrent versus integrated education and training. 
Proponents of sequential (as opposed to concurrent) programs argue that youths who are 
reading and computing at low levels get more out of training if their basic skills are improved 
before they enter occupational coursework.^^ The youths, they maintain, will have more 
choices of training and can get more out of the instruction.^^ Sequential programming also 
eases the burden of scheduling classes since students are freed from the pressure of 
simultaneously participating in two types of intensive coursework- Furthermore, their daily 
schedule can allow time for activities designed to address a variety of needs, such as life skills 
training, recreational activities, or part-time jobs. 

Advocates of "nonintegrated" programs (both sequential and concurrent) believe that 
employers value workers with a broad range of basic skills, not just those needed to master a 
specific occupation. In this view, workers with these more general skills will learn new skills 
more quickly on the job, be more flexible and productive than other workers in the long run, 
and be able more easily to shift to a new employer or even a new occupation when the 
economy changes and their original job no longer exists.^^ 



^^Hahn and Lerman, 1985. 

^"^Many existing occupational curricula require that entrants have tenth-grade reading and mathematics 
skills. 

^%ee, for example, National Academy of Sciences, 1984; Johnson and Packer, 1987; National 
Association of Manufacturers, 1982. 
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Advocates of concurrent - and especially integrated - programs focus on a different 
issue: the difficulty of keeping disadvantaged young school dropouts in programs. Students may 
find the education phase of a sequential program too much like their past high school 
experience, which many did not like, and may leave the program before they get to 
occupational skills training. In this view, being able to combine basic education with skills 
training - which has a more obvious connection to the job market - makes the education 
component more appealing.^^ It is argued, for example, that if students see that they need 
basic mathematics in order to make measurements for carpentry, they will be more motivated 
to solve addition and division problems. 

Another argument for concurrent programming - and especially integrated programs - 
rests on a narrower view of the purpose of basic skills education for young dropouts. Its 
proponents hold that instruction in basic skills should focus on the particular skills needed for 
a job (the goal of most young people in the program) rather than on imparting general 
knowledge that is less directly relevant to the lives of the youths .^^ 

2. Brokered versus in-house services. Practically speaking, brokering may be the 
only way that small agencies specializing in one type of service can provide multi-component, 
comprehensive programs. None of the small, community-based education providers 
participating in the demonstration, for example, had the capability to develop on-site training 
facilities offering a variety of training options. Agencies vnth a limited number of training 
courses might also choose to broker training for some participants in order to increase the 
range or quality of training available to them 

Brokered programs increase the operational challenges for the program operator, 
however There are potential difficulties, for example, in ensuring that participants in 
education will be accepted for training by other agencies, in scheduling the end of the 
education phase to coincide with a variety of different training schedules, and in monitoring 



^^Mathematica Policy Research, 1988. 

^^A final argument for concurrent programs rests on the particular features of most JTPA programs. 
Programs offering concurrent education and training tend to be shorter and do not involve a transition 
from education to training. If this transition also involves a shift from an education service provider to 
a second provider (for training), JTPA performance contracts may typically not allow final payment of the 
first provider unless participants receive a GED certificate, a milestone recognized under the JTPA 
performance standards. Since many youths with \yooT skills will not pass the GED test, this is a practical 
obstacle to sequential/brokered programs. 
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the progress of students referred to other agencies and the quality of the services provided to 
them. 

3. A further key dimension of local variation. JOBSTART sites operated in very 
different labor markets. The unemployment rates in the sites' metropolitan areas varied from 
a low of 3 percent in 1987 in Hartford, where CREC is located, to 12 percent in 1986 in 
Corpus Christi, where SER operated. Youth unemployment rates varied from 6 percent in 
1986 in Hartford to 27 percent in 1985 in New York City, where BSA was located.^^ 

Later chapters of this report present a more in-depth look at JOBSTART implementation 
and present generalizations about how local variation affected the way the program operated. 
However, as discussed in more detail in Chapter 2, caution must be exercised in making cross- 
site comparisons of program implementation and impacts. Since many features of programs 
differ among the sites, it is very difficult to isolate the influence of a difference in one faclor.^^ 

IV. The JOBSTART Evaluation and the Organization of This Report 

The evaluation of JOBSTART is divided into three main parts. The first dealt with the 
sites' implementation of the program. Launching JOBSTART, the initial report on the 
demonstration, discussed site selection and characteristics, the operation of the program within 
JTPA, and early implementation experiences.^ A second report, Implementing JOBSTART, 
completed the implementation analysis by describing the content of JOBSTART activities, the 
participation patterns of the young people in the program, and operational lessons to be drawn 
from the demonstration.^^ These findings are summarized and updated in this report. 

The second part of the evaluation is an analysis of program impacts; the early findings 
on impacts form the core of this report. The research was designed to separate out the effects 
of JOBSTART itself from events attributable to other factors (such as other services 
participants were receiving and events in their lives outside the program). To accomplish this, 



^^U.S. Department of Labor, Bureau of Labor Statistics, unpublished figures. 

^^e previous report (Auspos et al., 1989) also highlighted variations among the sites: whether 
they were serving JOBSTART youths in mainstream adult classes or in separate classes for youths; whetiicr 
they offered computer-assisted instmaion; and how they differed in scheduling (that is, the number of 
hours a day devoted to various activities) and in the length of their courses. Since these did not appear 
to have a major influence on program implementation or on participation by the youths, they were unlikely 
to have affected impacts and are not emphasized in this rejyort. 

^Auspos, 1987. 
Auspos et al., 1989. 

-27- 



all people who applied for JOBSTART and were found to be eligible were randomly assigned 
to either an experimental or a control group. Those in the experimental group were given 
access to the JOBSTART program services; those in the controj group were not, although they 
could receive other services offered in their community. Since the youths were assigned at 
random to the two groups, they were similar except for the fact that only the experimental 
group could receive JOBSTART services. 

Individuals in both groups were scheduled to be surveyed 12, 24, and approximately 48 
months after being randomly assigned to their group. (The time frame for applying to 
JOBSTART varied from site to site but ranged overall from August 1985 through November 
1987. Hence the fielding of each wave of the survey also extended over many months.) Using 
these surveys, the experiences of the two groups can be compared to estimate the effect of the 
program on educational attainment, employment, earnings, use of public benefits, and other 
outcomes. 

The third part of the evaluation will assess the cost-effectiveness of the program. While 
the present report includes a discussion of the costs of JOBSTART, the final report, based 
on four years of follow-up, will summarize the findings on program costs and benefits. 

As discussed above, this research design uses random assignment to provide reliable 
information on the central question - the impact of JOBSTART services - by comparing the 
experiences of two groups that were equivalent except for the experimental group's access to 
the program. It is important to understand that it was not designed to answer certain other 
questions with equal rigor. Most important, the study was not designed to rigorously compare 
the effectiveness of sequential versus concurrent or integrated programs. The young people 
were not randomly assigned to these different programmatic approaches. Instead, the sites 
offering these three approaches operated in different kinds of settings and local labor markets 
and served different types of youths with varying backgrounds and interests. Thus, the 
structure of the program was not the only difference among sites. Given these many 
differences, it is impossible to isolate the influence of one factor - such as concurrent versus 
sequential program structure on program effectiveness.^^ 



rigorously compare the impacts of different programmatic approaches, more than one approach 
would have to be operated in each site and youths would have to be randomly assigned to one of them. 
Even with this design, if the programs differed on several dimensions, it would still be impossible to 
isolate the effect of any one dimension. This type of research has rarely been undertaken. Examples 

(continued...) 
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Chapter 2 of this interim report on JOBSTART presents the key research questions and 
the research design used to address them. It also includes information showing that the sites 
did succeed in recruiting the young, disadvantaged target group of the demonstration. Chapter 
3 discusses the implementation of the JOBSTART program, concluding that most youths in 
JOBSTART participated in education and training activities more than the typical participant 
in JTPA-funded programs and nearly as much as the typical Job Corpsmember. Here, as in 
all subsequent chapters, the report seeks to understand the overall results by examining 
whether and how JOBSTART ope'-ated differently for key subgroups of youths in the sample, 
with a particular focus on young men and young mothers. Chapter 4 examines educational 
outcomes, especially the degree of JOBSTART youths' participation in education and training 
compared to that of the control group, and whether JOBSTART led to increased attainment 
of a high school diploma or GED during the two years of follow-up. Chapter 5 examines the 
early indications of how this investment in "human capital" affected youths' employment, 
earnings, welfare receipt, and other outcomes. In Chapter 6, the analysis compares the 
experiences of youths in the 13 sites in the demonstration and - to the extent possible given 
the research design of the study - explores possible explanations for differences among sites. 



"(...continued) 

include MDRCs study of the impacts of job search alone versus job search plus community work 
experience in San Diego (Goldman et al., 1986) and Mathematica Policy Research's study of alternative 
reforms of the unemployment insurance system. 
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CHAPTER 2 



STUDY DESIGN AND SAMPLE CHARACTERISTICS 

This chapter presents the study design used to address the research questions posed in 
Chapter 1, and the characteristics of the research Si. -^ple used in the analysis. Part I describes 
the research approach, with special emphasis on the random assignment research approach for 
assessing the difference the program makes in the lives of young people. Part 11 describes the 
research sample for this report. 

L An Overview of the Study Dcsigp 

Although education and training services for young school dropouts are limited, some 
youths who entered JOBSTART would have gotten GEDs or high school diplomas, found jobs, 
increased their earnings, or gotten off welfare on their own even if they had not been in the 
program. As noted in Chapter 1, to isolate the impact of JOBSTART from other factors that 
may produce such outcomes, MDRC randomly assigned applicants to experimental and control 
groups. The two groups were similar except that only the experimental group could receive 
JOBSTART services. Comparison of the two groups* experience^? during the two years after 
random assignment (the follow-up information available for this report) provided a reliable 
estimate of the difference the program made during an early post-program period.^ 



^Sources of the data for the evaluation are discussed in deuil in Appendix A. They include 
enrollment forms completed just prior to random assignment; a management information system (M13) 
that provided data on participation in the program; results from the Test of Adult Basic Education 
(TABE) administered to members of the experimental group; follow-up surveys (for this report) 
conducted 12 and 24 months after random assignment and (for the final report) approximately 48 
months after random assignment; program cost data from a variety of sources; and qualitative data based 
on interviews with the program staff, field observations of program operations, and focus group 
discussions with participants. 
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A. How Random Assignment Was Carried Out 



Figure 2.1 shows the steps in the intake and random assignment process/ Youths who 
expressed an interest in program services entered the program through a process that took 
from cne day to one month (10 days on average), depending on the site.^ Most of the steps 
were part of the usual JTPA Title IIA (or, in Atlanta, Los Angeles, and Phoenix, the Job 
Corps) intake procedure; in most sites, only the reading test and random assignment were 
added for the JOBSTART Demonstration. 

The order of steps varied from site to site, as did the division of responsibility between 
the program operator and the local service delivery area (SDA). The process included: 

• CHent recruitment JOBSTART was voluntary, so the program operator 
and, in some cases, the SDA actively recruited youths to apply, using a 
variety of techniques to meet their enrollment goals. Program staff 
approached potential recruits through media announcements; mailings to 
dropouts and welfare recipients; and outreach visits to schools, parks, and 
other youth gathering places. They distributed posters and fliers advertising 
program services and sought referrals of eligible youths from JTPA, 
community organizations, schools, and social service agencies. Recruitment 
activities frequently took staff beyond the boundaries of the office and the 
nine-to-five workday. Recruitment through public school referrals or 
outreach was productive in school-based JOBSTART programs. 

• Informational interview. In a brief interview, JOBSTART staff explained 
to potential applicants the program's services and obligations and, often, 
the random assignment procedures. Some sites also regularly included a 
tour of their facilities to help recruits understand program services, 
opportunities, and demands. 

• Assessment Program staff assessed whether applicants met the age (17 
to 21), educational status (school dropout), and income requirements for 
JOBSTART. They also ascertained the youths' support service needs and 
appropriateness for the program, screening out those with problems the 
program was not equipped to handle. The assessment process was 
relatively extensive at the Job Corps sites, which had the broadest array of 
support services. Job Corps staff assessed recruits for emotional problems, 
drug and alcohol abuse, trouble with the law, unstable living situations, 
health problems, and motivation. Other sites screened mostly to identify 



^ee Auspos et al., 1989, for the details. 

^Sites varied greatly in the amount of initial assessment they conducted before allowing entry into 
the program. There was also wide variation in state and local interpretation of the documentation 
needed to establish eligibility for JTPA Title IIA programs. 
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youths who were likely to prove dangerous or disruptive, such as those 
with evident drug or alcohol problems. 

• Reading test Most program operators tested the reading level of recruits 
early in the intake process to determine that applicants read below the 
eighth-grade level, as required by JOBSTART eligibility criteria. Four sites 
(the three Job Corps Centers and CET/San Jose) delayed testing until later 
in the program, limiting their testing to participants. As noted earlier, sites 
were permitted to enroll up to 20 percent of their recruits with higher 
leading scores to help meet enrollment goals. Some sites also set a lower 
limit or floor - a fourth-, fifth-, or sixth-grade reading level. These 
program operators felt that the youths would need to read at least at these 
levels in order to benefit from the education and training services that 
were available locally. 

• JTPA Title IIA/Job Corps certification. Recruits had to prove that they 
fulfilled eligibility criteria for JTPA Title IIA-funded services. At the Job 
Corps sites, recruits also had to meet Job Corps eligibility criteria. At all 
the sites, certification of eligibility required proof of residency, age, and 
economic disadvant e. SDAs at most sites required applicants to provide 
supporting docun.wntation of all aspects of JTTA Title HA eligibility for 
approval of enrollment into JOBSTART. Local regulations and practices 
affecting the certification process strongly influenced the speed and ease 
of certification. JTPA Title IIA certification procedures were cited by 
program operators in six sites as a major bottleneck in the intake and 
enrollment process. 

• Informed consent form, enrollment form, and random assignment After 
staff described the random assignment process, the applicant signed an 
informed consent form, agreeing to accept the results of random 
assignment and to cooperate in follow-up survey interviews. Program or 
SDA staff then filled out the enrollment form, using information provided 
by the applicant. Staff then telephoned MDRC, where random assignment 
was made. Youths entering the experimental group were told to report 
to classes or, in some sites, to an orientation session. Program staff 
contacted experimental who did not appear for program activities, 
encouraging them to participate and assisting them with needed support 
services. Applicants assigned to the control group were reminded that they 
were part of the research project and would be contacted later. They were 
also told that they could seek services elsewhere on their own. 
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A total of 2,312 people were randomly assigned: 1,163 to the experimental group and 
1,149 to the control group.^ Sites conducted random assignment over varying periods of time. 
Connelley Skill Learning Center enrolled the first sample members in August 1985, and the 
Los Angeles Job Corps enrolled the last sample members in November 1987. Open- 
entry/open-exit sites continuously recruited applicants to maintain enrollment levels, while sites 
operating fixed-cycle programs - such as Connelley in Pittsburgh, Chicago Commoas, and 
SER/Corpus Christi - intensified recruitment efforts before the start of classes. 

Overall, sites reported that about 89 percent of the youths in the experimental group 
participated to some extent in JOBSTART. The percentage participating did vary among the 
sites, from a high of 100 percent at Allentown in Buffalo and El Centro in Dallas to a low of 
64 percent at CET/San Jose. (Site-specific participation information is presented in Chapter 
3.) Four factors influenced the percentage of experimental reported to be active in the 
program: 

• Length of the intake process. The process of selection into the 
JOBSTART Demonstration took a relatively short time in many sites, often 
less than a week. However^ at a few sites, the extended checks of 
eligibility (most important in the Job Corps sites) meant that intake lasted 
much longer^ and in the process some youths who were eventually assigned 
to the experimental group found other program options or lost interest. 

• Opcn-entry/open-exit versus fixed-cycle scheduling. Open-entry programs 
allow young people to enter and finish at any time^ while other programs 
operate on fixed schedules of class cycles.^ Youths assigned to the 
experimental group in fixed-cycle sites might face delays in program start- 
up, resulting in lower participation rates. 

• Start-up or schedulinc problems. Some sites had unexpected problems 
getting youths into services. The most notable example was the experience 
of the early entrants at CET/San Jose, where program slots were not 
available for up to a month after random assignment because of funding 
cuts. This delay contributed to this site's experimentals having the lowest 
rate of participation in JOBSTART services. 



"^All but one of the 2312 youths who were randomly assigned filled out enrollment forms providing 
pre-program baseline data on age^ sex, prior employment, extent of schooling, and other characteristics 
used in impact calculations and to define key groups within the full sample. 

^Sites operating open-entry/open-exit programs included Allentown in Buffalo, the Atlanta Job 
Corps, CET/San Jose, the East Los Angeles Skills Center, El Centro in Dallas, the Los Angeles Job 
Corps, and the Phoenix Job Corps. EGOS in Denver offered classes on a semester schedule but allowed 
entry whenever classes were in session. 
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* Differences in sites^ attendance reporting. The program elements counted 
in participation in all sites included education, training, and other activities 
such as life skills training, work eTcperience, and - in the Job Corps sites 
- a lengthy orientation. Participation in an extended assessment of 
training interests at CET/San Jose was not included in reported houn. 
Therefore, if youths attended this assessment and nothing else, their 
reported hours were zero and they were counted as nonparticipants. This 
could have affected CET/San Jose's participation rate and reported hours 
in activities. 

B. The Research Samples Used in This Report 

Follow-up surveys at 12 and 24 months after random assignment gathered data on 
outcomes such as participation in education and training programs, educational attainment, 
employment, earnings, and use of public benefit programs.^ Of the 2,311 youths in the full 
research sample, 1,839, or 80 percent, provided 24 months of survey follow-up data and 
constitute the "impact sample" analyzed in this report in Chapters 4 through 67 The 949 
experimental in this sample are used to examine the implementation of JOBSTART, in 
Chapter 3, where issues such as participation rates in JOBSTART and its components, and 
hours and duration of participation, are examined. 

C. Key Methodological Issues for the Impact Analysis 

For this study to produce unbiased estimates of program impacts, several conditions must 
be met. These are addressed in the following questions:^ 

1. Did random assignment lead to a group of experimental with the same 
measured pre-program characteristics as the controls? Random assignment - properly 
implemented - creates a group of JOBSTART controls with the same pre-program 
characteristics as JOBSTART experimentals, so that observed differences between 



^As noted in Appendix A, information on participation in JOBSTART was provided by the siies 
as part of a special management information system (MIS) created for the demonstration. The follow- 
up surveys collected information on participation in all other education, training, and employment 
programs for both experimentals and controls. 

^Most responded to both the 12- and 24-month follow-up surveys (1,604, or 87 percent of 
responders), while the remainder (235 people) responded to a special combination survey covering the 
entire 24 months, which was fielded for youths who did not respond to the 12-month survey but were 
located at 24 months. 

®For a fuller discussion, see Appendix B. 
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experimental and controls in post-random assignment behavior provide unbiased, accurate 
estimates of program impacts.^ The information presented in Appendix B (Table B.l) for the 
2,311 people randomly assigned shows that there were virtually no measured differences in 
characteristics between experimentals and controls. 

2» Do experimentals and controls in the impact sample for this report (that is, 
those with 24 months of !»urvey follow-np) have the same measured pre-program 
characteristics? Appendix Table B.2 shows that the 1^839 experimentals and controls in the 
impact sample are virtually identical in average measured characteristics. 

3. Are those 1,839 sample members with 24 months of survey data representative 
of the entire JOBSTART sample of 2^11? Twenty-four months of survey data are available 
for nearly 80 percent of all the youths who were randomly assigned. Appendix Table B.3 
shows that there are some statistically significant differences between those who responded to 
the surveys and nonresponders. Responders were more likely to be experimentals: 82 percent 
of experimentals provided 24 months of survey follow-up information compared to 78 percent 
of controls. Responders were also more likely to have entered the sample at the Allen town 
in Buffalo site, to have had no criminal convictions between age 16 and baseline, and to have 
lived with both parents at age 14. 

When nonresponse is randomly distributed among members of both the experimental and 
control groups, it is troublesome only because it reduces the sample size and thus the statistical 
power to find impacts of a given size.^^ However^ when nonresponse is greater among one 
research group (such as controls) or among members of either research group who have 
certain characteristics (such as those who, at age 14, lived with their parents), impacts may be 
biased slightly unless they are corrected for nonresponse. 



^is condition is known as the "internal validity" of the estimate. 

^^e only difference that was statistically significant at the 5 percent level was that experimentals 
in the sample were slightly more likely than controls to be a part of an AFDC case headed by another 
member of their household. On a site-by-site basis, the 24-month JOBSTART impact sample consists 
of 26 separate groups of experimentals and controls. If experimentals in any site are compared with 
controls in that site, the internal validity of site impacts may be assessed. As would be expected in 13 
relatively small subsamples of the full 24-month impact sample of 1,839, there are a few 
experimental-control differences in demographic characteristics within individual sites. 

^^Randomly distributed nonresponse does not alter the expected values of adjusted mean outcomes, 
and thus does not bias impacts. 
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The impacts presented in this report do not include any corrections for the differences 
in survey responders and nonresponders just described. The success of attempts to 
implement such corrections is uncertain, and the differential response rates found do not seem 
large enough to warrant such measures, which could introduce biases of their own. The high 
overall response rate of 80 percent makes fmdings from the 24-month impact sample 
representative of a very broad group of the full sample. 

4. Did most experimentals receive JOB START services, and did relatively few 
controls receive them or any equivalent services? For this condition to be met, experimentals 
must participate in JOBSTART, and controls must be excluded from JOBSTART and not find 
equivalent services elsewhere in their community. As discussed above, nearly 90 percent of 
experimentals were active in JOBSTART. For controls, random assignment procedures were 
followed, and virtually no controls were served in JOBSTART programs. In addition. Chapter 
4's analysis of the receipt of education and training services from all sources indicates that 
controls did not find an equivalent level of services elsewhere. For example, in the first year 
after random assignment, 91 percent of experimentals and 29 percent of controls participated 
in some type of education or training activity. As this and the other measures used in Chapter 
4 indicate, experimentals did receive a noticeably greater total amount of employment and 
training services. But it is important to keep in mind that controls were not an unserved 
group; many received substantial services from sources other than the JOBSTART programs. 
The impact findings presented in Chapters 4 through 6 of this report, therefore, should be 
interpreted as measuring the incremental impact of the services received by experimentals 
above the level of services received by controls. 

5. Do the impacts per person assigned to the experimental group differ greatly 
from the impacts per person participating in J0BSTAR17 Some of those who were 
randomly assigned to the experimental group (the group given access to the JOBSTART 
program) never participated. However, they were still included as part of the experimental 



most flexible correction for nonresponse is incorporation of an additional equation for survey 
response into a two-equation system with the impact equation. 
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group when average impacts were calculated, somewhat Vatering down" the impacts. 
Fortunately, the percentage of nonparticipants was small (only 11 percent of the 949 
experimentals in the impact sample), so including them "diluted" the impacts only slightly. In 
other words, while the impacts refer to all surveyed experimentals (nonparticipants as well as 
participants), they would be only slightly higher if they were adjusted to apply to surveyed 
participants only.^'^ 

11. The Characteristics of the .TOBSTART Youths 

Examining the pre-program experiences and characteristics of the young people in the 
JOBSTART sample is important for three reasons. First, it shows whether the sites in the 
demonstration succeeded in enrolling economically disadvantaged young people with poor skills 
who were the target group for the demonstration. Second, it permits a comparison of the 
JOBSTART youths with those served by other important employment and training programs. 
Third, much of the analysis in this report moves beyond results for the full sample of 
JOBSTART youths to examine whether and how the program worked differently for subgroups 
of young people (especially young males, mothers, arid other women), and understanding the 
pre-program characteristics of these groups is the first step in this analysis. 

The thiid point is important because groups defined by a single characteristic (such as 
gender, age, prior employment, or the type of site to which they applied) may vary on other 
characteristics as well. Young women in the sample, for example, may have h»H less prior 
employment experience and more prior public assistance receipt than did the men in the 
sample. Sites offering education followed by training at another agency may attract very 
different applicants than those known for their training courses. Understanding the 
combination of characteristics associated with subgroups such as men or women or types of 
sites helps prevent misinterpretations of any observed differences in program participation and 



^^If the nonparticipants had not been counted, the experimental group would no longer have been 
truly comparable to the control group. Including them in the impact calculations was designed to avoid 
a form of "selection bias" - in this case, caused by those who had "selected themselves" out of their 
chance to join the JOBSTART program or were discouraged by program staff. 

^''See Appendix B for details on such adjustments. In some sites, nonparticipation rates were 
considerably higher than the 11 percent for the entire impaa sample, so the difference between impacts 
per experimental and per participant is greater. 
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effectiveness. With this goal of the report in mind, Part II of this chapter summarizes the 
characteristics of the youths at each site and of key subgroups of young people that will be 
examined throughout the remaining chapters. 

A. The Characteristics of the Impact Sample of .TOBSTART Youths 

Table 2.1 provides detailed background information on the impact sample of JOBSTART 
youths and shows that the sites in the demonstration succeeded in recruiting the intended 
target group. The column labeled "all sites" shows the characteristics of the entire impact 
sample; the remaining columns are discussed in Part IIB of this chapter. The sample is made 
up of slightly more women than men; most of the sample are members of minority groups and 
are unmarried; nearly three-fourths are under 20 years of age; slightly fewer than half did not 
work during the year prior to random assignment; and about 60 percent left school before the 
eleventh grade. 

JOBSTART participants appear to have been more disadvantaged than the majority of 
youths served nationwide by JTPA Title IIA programs during the period JOBSTART was in 
operation. In the effort to serve those youths at risk of chronic unemployment, JOBSTART 
worked exclusively with dropouts, a segment of the youth population that makes up a relatively 
small part of JTPA Title IIA enroUees. Even when the comparison of participants is limited 
to young dropouts, it appears that JOBSTART reached a more disadvantaged population than 
did most JTPA Title HA-funded programs.^'' 



^ More detailed comparisons are included in Appendix C. 

^^e only real divergence from the intended target group occurred because a slightly higher than 
planned percentage of the youths read at the eighth-grade level or above. This happened because of 
the educational testing practices of some sites and is not shown in Table 2.1 because of inconsistent data 
among the sites. Some sites did not administer the reading test as part of the initial assessment of 
suitability for JOBSTART, which occurred before random assignment. 

^^Approximately 56 percent of JOBSTART participants were receiving some form of public- 
assistance at the time they entered the program, compared to 39 percent of young dropouts served by 
JTPA Title IIA programs. Moreover, the proportion of JOBSTART participants who received AFDC 
(38 percent) was much higher than that of young dropouts in other JTPA Title IIA programs (21 
percent). This higher rate of welfare receipt partly reflects the fact that a greater proportion of 
JOBSTART participants were young women (53 percent) compared to the dropout group participating 
in other JTPA Title IIA programs (45 percent females). Also, minorities were much more heavily 
represented in JOBSTART than in JTPA Title IIA-funded services for young dropouts nationally. 
Hispanic dropouts constituted 44 percent of JOBSTART participants but only 14 percent of JTPA Title 
IIA dropouts, and JOBSTART served proportionally more black dropouts (46 percent) than did other 
JTPA Title IIA programs (34 percent). 
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TABLE 2^1 

CHARACTERISTICS AT THE TIME OF RANDOM ASSIGNMENT, BY SITE 



Concurrent 



o 
I 



CharactcrUtic 
and Subgroups 



Gender 
Women 
Men 

Ethnicity 
White, non-Hispanic 
Black* non«Hispbnic 
Hispanic 
Other 

Ethnicity, by gender 
Women 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 

Other 
Men 

White, non-Hispanic 
Black, non-Hispanic 
Hi spanic 
Other 

Parental status 
Women living with own child(ren) 
No 
Yes 

Men who have own child(ren) 
No 
Yes 

Employed within past year 
No 
Yes 

Prior employment, by gender 
Women employed within past year 
No 
Yes 

Men employed within past year 
No 
Yes 

Left school in grade 11 or 12 
No 
Yes 



ERLC 



Sample size 













East 






cm i 

SER/ 




All 

All 


Atlanta 


CtT/ 


Ciiicogo 


Conneucy 


LA Skills 


EGOS 


Phoenix 


Corpus 




1 Sites 


Job Corps 


San Jose 


Conmons 


(Pittsburgh) 


Center 


(Denver) 


Job Corps 


Christi 




52.6%*** 


59.0 V 


49.3% 


43.2^ 


53.3% 


44.0% 


64.5% 


51.5% 


39.4% 




47.4 


41.0 


50.7 


56.8 


46.7 


56.0 


35.5 


48.5 


60.6 




o . ^ 


J.J 




4 1 


8.2 


1.0 


11.5 


23.1 


8.1 




45.7 


96.7 


4.6 


81.1 


91.8 


0.0 


30.6 


17.7 


4.7 




42.6 


0.0 


73.7 


14.9 


0.0 


95.0 


55.2 


53.8 


87.3 




3.3 


0.0 


9.2 


0.0 


0.0 


4.0 


2.7 


5.4 


0.0 




4.6*** 


3.3 


7.2 


2.7 


3 . :i 


1.0 


5.5 


12.3 


3.8 




24.2 


55.7 


2.0 


32.4 


50.0 


0.0 


23.5 


4.6 


3.0 




22.5 


0.0 


38.8 


8.1 


0.0 


41.0 


33.3 


32.3 


32.6 






n n 
u • u 


I . J 


n n 


0 n 


? 0 


7 7 


2.3 


0.0 




3.9 


0.0 


5.3 


1.4 


4.9 


0.0 


6.0 


10.8 


4.2 




21.5 


41.0 


2.6 


48.6 


41.8 


0.0 


7.1 


13.1 


1.7 




20.1 


0.0 


34.9 


6.8 


0.0 


54.0 


21.9 


21.5 


54.7 




1.9 


0.0 


7.9 


0.0 


0.0 


2.0 


0.5 


3.1 


0.0 




26.3*** 


29.5 


42.8 


18.9 


19.0 


32.0 


27.9 


23.1 


16.9 




26.3 


29.5 


6.6 


24.3 


34. Z 


12.0 


36.6 


28.5 


22.5 




41.6 


39.3 


45.4 


39.2 


37.0 


52.0 


33.3 


43.1 


51.7 




5.9 


1.6 


5.3 


17.6 


9.8 


4.0 


2.2 




8.9 




47.3*** 


37.7 


36.8 


54.1 


28.8 


55.0 


39.9 


55.4 


33.5 




52.7 


62.3 


63.2 


45.9 


71.2 


45.0 


60.1 


44.6 


66.5 




29 7^^^ 








Ifl 5 


7R 0 


31 7 

J 1 . / 


36.2 


21.6 




22!9 


29.5 


27.0 


16.2 


34.8 


16.0 


32.8 


15.4 


17.8 




17.6 


8.2 


14.5 


27.0 


10.3 


27.0 


8.2 


19.2 


11.9 




29.8 


32.8 


36.2 


29.7 


36.4 


29.0 


27.3 


29.2 


48.7 




58.6*** 


59.0 


38.8 


43.2 


64,1 


69.0 


55.2 


60.8 


75.0 




41.4 


4) 0 


61.2 


56.8 


35.9 


31.0 


44.8 


39.2 


25. 0 




1,839 


61 


152 


74 


184 


100 


183 


130 


236 





jcontinued) 



TABLE 2.1 (continued) 



Seqv entlal/Brokered 







El 




Allen- 








Al 1 


ttCil k 1 u 




town 




CREC 


onu juu9 roups 


1 J oi vCS 


\ Da 1 1 Q S ^ 


Corps 


^ DU T TO 1 




\ naiL Turu / 


benaer 














Womfn 


52.6%*** 


54.2% 


59.6% 


58.6% 


44.5% 


64.4% 


Hen 


47.4 


45.8 


40.4 


41.4 


55.5 


35.6 


Ltnn 1 ci Zj 














Wni te^ non-Hi spAni c 


o 


b • 3 


J .z 




'i A 


A £ 

9 .b 


Dltt^U MAM L|{«MAm1« 

Di ACK^ non-Hi spAni c 


Aft 7 


7n 1 


3U .U 


77 1 


A7 9 


3D . J 


HI Spam c 


AO C 
• D 


Z 1 .9 


js .s 


7 O 




. 1 


utner 


J • J 


1 1 


19 A 


n 7 






Ethnicity, by gender 














Women 














Wnite, non'-Hispanic 




^ • D 


1 A 




1 7 

1 ■ / 


9 1 


Black, non-KispanIc 


24!2 


34.2 


28.9 


43.6 


26.1 


32.2 


Hispanic 


22.5 


15.5 


22.9 


5.0 


16.8 


29.9 


Other 


1 A 
1 






n 7 




U .U 


Men 














White, non-Hispanic 


3.9 


1 n 

1 .9 


1 Q 
1 .0 


3 .U 


1 . / 


C. .J 


Black, non-Hispanic 


11 a 
l \ *b 


Jb • 1 


11 1 


J J .b 


A \ 0 


Z<f . 1 


Hi spanic 


• 1 


b .3 


lie 


9 Q 


I 1 A 

I I .o 




Other 


1 . 9 


1 .3 


D.O 


n n 
U .U 


n Q 
U .o 


n n 
U .U 


Parental status 














Women living witn own cniia^rcn^ 














No 


26 .3*** 


21.3 


on k 


OC 7 

Zb . / 


Zo .b 


1£ 0 

Jb .0 


Yes 


26.3 


32.9 


30.3 


32.9 


16. D 


27.6 


Men who have own child(ren) 














nO 


S 1 • D 


A\ Q 


IQ A 


in n 




11 0 

J 1 .V/ 


Tes 


3 • O 


1 Q 


n Q 


1 1 A 


1.7 


A fi 


Employed within past year 














y A 
No 


A7 1*** 


S3 . c 


f 3 . c 


R7 U 


«A 7 


11 n 

J 1 . U 


Yea 




3<| . U 




AO 1 
S c . 1 


j3 . J 




Prior employment, by gender 














woiren empioyea witnin past year 














y A 
No 


9Q 7*** 


LI % 1 


47 ? 


J7 • J 


74 1 


Yes 


22.9 


26.5 


12.4 


19.3 


15.1 


40.2 


Men employed within past ytar 














No 


17.6 


17.4 


28.0 


18.6 


35.3 


6 9 


Yes 


29.8 


28.4 


12.4 


22.9 


20.2 


28.7 


Left school in grade 11 or 12 














No 


58.6*** 


67.7 


41.3 


53.6 


60.5 


74.7 


Yes 


41.4 


32.3 


58.7 


46.4 


39.5 


25.3 


Sample size 


1,039 


155 


218 


140 


119 


87 
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TABLE 2*1 (continued) 



CharacterUtf c 
and Subgroups 



Received occupational 
training within past year 

No 

Yes 



Concurrent 



East SER/ 
All Atlanta CET/ Chicago Connelley LA Skills EGOS Phoenix Corpus 

13 Sites Job Corps San Jose Commons (Pittsburgh) Center (Denver) Job Corps Christi 



83.1%*** 65.6% 90.1% 90.5% 66.8% 86.0% 91.8% 94.6% 69.5% 

16.9 34.4 9.9 9.5 33.2 14.0 8.2 5.4 30.5 



Age 

16-19 
20 or 21 



73.9*** 77.0 78.9 47.3 54.9 78.0 76.5 86.2 70.3 

26.1 23.0 21.1 52.7 45.1 22.0 23.5 13.8 29.7 



Marital status 
Ever married 
Never married 



9.5*** 6.6 12.5 1.4 3.8 4.0 7.1 10.8 29.7 

90.5 93.4 87.5 98.6 96.2 96.0 92.9 89.2 70.3 



I 

I 



Living in own household or 
with boy/girlfriend 

No 

Yes 

Own AFDC case or receiving 
General Assistance 

No 

Yes 



81.6*** 88.5 
18.4 11.5 



73.1*** 67.2 
26.9 32.8 



87.5 
12.5 



90.8 
9.2 



78.4 
21.6 



47.3 
52.7 



75.5 
24.5 



56.5 
43.5 



90.0 
10.0 



76.0 
24.0 



77.0 
23.0 



73.2 
26.8 



85.4 
14.6 



83.1 
16.9 



76.3 
23.7 



86.9 
13.1 



Own AFDC case 
No 
Yes 



78.6*** 78.7 94.7 73.0 68.5 79.0 74.9 83.8 88.6 

21.4 21.3 5.3 27.0 31.5 21.0 25.1 16.2 11.4 



Receiving Food Stamps 
No 
Yes 



62.2*** 63.9 90.1 39.2 25.5 68.0 58.5 76.2 68.6 

37!8 36.1 9.9 60.8 74.5 32.0 41.5 23.8 31.4 



ST) 



ERiC 



Arrcste<< since age 16 
No 
Yes 

Lived with both parents at 
age 14 

No 

Yes 

Sample size 



85.2*** 
14.8 



1,839 



88.5 
11.5 



65.1*** 75.4 
34.9 24.6 



61 



76.3 

23.7 



52.0 
48.0 



152 



81.1 
18.9 



74.3 
25.7 



74 



89.7 
10.3 



79.9 
20.1 



84.0 
16.0 



54.0 
46.0 



83.1 
16.9 



59.6 
40.4 



184 



100 



183 



87.7 
12.3 



46.2 
53.8 



130 



78.8 
21.2 



45.3 
54.7 



236 
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TABLE 2.1 (continued) 



Characteristic 
and Subgroups 



Received occupational 
training within past year 

No 

Yes 



Sequential/In-House 

El Allen- 
All Centro lA Job town 
13 Sites (Dallas) Corps (Buffalo) 



Sequential/Brokered 



83.1%*** 
16.9 



69.0% 
11.0 



91.7% 

e.3 



82.9% 
17.1 



BSA 
(NYC) 



CREC 
(Hartford) 



79.8% 
20.2 



82.8% 
17.2 



Age 
16-19 
20 or 21 



73.9*** 
26.1 



83.9 
16.1 



77.5 
22.5 



73.6 
26.4 



73.9 
26.1 



80.5 
19.5 



Marital status 
Ever married 
Never married 



9.5*** 
90.5 



11.6 
88.4 



5.0 
95.0 



4, 
95, 



1.7 
98.3 



5.7 
94.3 



Living in own household or 
with boy/girlfriend 

No 

Yes 



81.6*** 
18.4 



91.6 
8.4 



86.7 
13.3 



59.3 
40.7 



95.0 
5.0 



77.0 
23.0 



Own AFDC case or receiving 
General Assistance 

No 

Yes 



73.1*** 
26.9 



84.5 
15.5 



67.0 
33.0 



52.1 
47.9 



77.3 
22.7 



70.1 
29.9 



Own AFDC case 
No 
Yes 



78.6*** 
21.4 



84.5 
15.5 



69.3 
30.7 



67.9 
32.1 



83.2 
16.8 



73.6 
^6.4 



Receiving Food Stamps 
No 
Yes 



62.2*** 
37.8 



65.8 
14.2 



69.7 
30.3 



30.0 
70.0 



62.2 
37.8 



6M 
37.9 



Arrested since age 16 
No 
Yes 



85.2*** 
14.8 



90.3 
9.7 



90.4 
9.6 



87.1 
12.9 



89.9 
10.1 



80, 
19. 



Lived with both parents at 
age 14 

No 

Yes 



65.1*** 
34.9 



71.6 
28.4 



75.2 
24.8 



80.7 
19.3 



70.6 
29.4 



79.3 
20.7 



Sample size 



1.839 



155 



218 



140 



119 



87 
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TABLE 2*1 (continued) 



SOURCE: MORC calculations frow JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all 1,839 sample members for whom there were 24 months of follow-up survey dat 
Sample sizes reported may fall short of this number because of Items missing from some sample members* questionnaires. 
Distributions may not total 100.0 percent because of rounding. 

A Pearson chi-square statistic was used to test the hypothesis of equal distributions. Statistical significance levels 
Indicated as ■ 1 percent; - 5 percent; * - 10 percent • 



Nationwide, 80 percent of Job Corpsmembers were school dropouti; in program year 
1986, when the JOBSTART Demonstration was in operation, but their other characteristics 
suggest greater barriers to employment than the JOBSTART youths faced. Job Corpsmembers 
tended to be younger than JOBSTART participants: 42 percent were age 16 or under in 1986 
compared to 29 percent in JOBSTART, Sixty-one percent read at the sixth-grade level or 
below at entry into the Job Corps compared to 52 percent in JOBSTART. On the other 
hand, a higher proportion of JOBSTART participants were receiving public assistance and were 
members of minority groups than were Job Corpsmembers. The residential character of the 
Job Corps program also introduces another difference: All residential Corpsmembers are willing 
and able to live away from home, but an unknown - though probably large - portion of 
JOBSTART members would not fall into this category. 

These comparisons suggest that JOBSTART sites did succeed in attracting disadvantaged, 
young school dropouts, as intended in the demonstration. However, these young people were 
not among the most disadvantaged youths: Relatively few reported at program intake that they 
had criminal arrests; most were not teenage parents; and about half had worked during the 
year before random assignment.^ In summary, the JOBSTART youths probably fell between 
the typical JTPA and Job Corps participant in initial skills levels and job readiness. 

B* Site Differences in Sample Characteristics 

When individuals with certain characteristics are largely concentrated in one or a few 
sites, the influences of their individual characteristics on program implementation and impacts 
are "confounded" with the influence of site characteristics. This is virtually a non-issue with 



^^In JOBSTART, JTPA performance standards and practices led some JTPA Title llA-funded sites 
to exclude youths with very low reading scores; the Job Corps sites in JOBSTART appeared to include 
a higher proportion of youths with very low reading scores than did other sites. 

^^e difficulties encountered in implementing an unsuccessful random assignment study of the 
residential versus nonresidential Job Corps illustrate the importance of this difference. The study 
originally assumed that a substantial portion of the Job Corps applicant pool would be indifferent as 
to whether they got into a residential or nonresidential program, and the study proposed to randomly 
assign members of this group to the two program types. This "indifferent** group turned out to be too 
small a portion of applicants for the study to proceed. 

^It is very likely that youths underreported past arrests, since they were asked about this at 
program intake and may have assumed that a positive response would lower their chances of getting into 
the program. Also, they may not have wanted to provide this information to staff, whom they did not 
yet know. In addition, only 106 men (or 12 percent of men) report being a father. 
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regard to the proportion of experimentals and controls at the sites: All sites had approximately 
equal proportions of the two groups. Not so with ethnicity, in particular: The proportion of 
black sample members ranged from zero at the East Los Angeles Skills Center to 97 percent 
at the Atlanta Job Corps, as shown in the individual site columns of Table 2.1. Thus, while 
the proportion of experimentals and controls is almost independent of site, the influence of 
ethnicity is much more confounded with the influence of site characteristics.^^ Because of 
this, simple experimental-control comparisons of post-program outcomes for the full sample can 
confidently be interpreted as resulting from experimentals' access to JOBSTART, but there is 
not the same confidence about comparisons of experimental-control experiences for ethnic 
subgroups (which could be heavily influenced by different program structures, labor market 
conditions, or other important background factors completely external to JOBSTART). Most 
characteristics lie somewhere between independence and confounding. Thus, unless special 
techniques are used to remove associations between site and other characteristics, impact 
comparisons for many subgroups and site groupings may be misleading. 

Fortunately, in view of the heavy eniphasis this report places on comparisons of outcomes 
by gender, there is much less cause for concern in making comparisons of impacts for gender- 
defined subgroups than for those based on ethnicity. The sample was 47 percent male overall, 
and men were distributed across sites much more evenly than were blacks or Hispanics.^^ A 
more refined analysis of gender, appearing throughout this report, splits the sample further, 
based on parenting status. This creates categories that can signal differences in barriers to 
employment or willingness to sacrifice in order to obtain a steady source of earned income. 
Overall, half of the women lived with children of their own, and half did not; among men, 
about one-seventh reported that they already were parents at baseline, and six-sevenths 
reported having no children at that time. The proportions of parents among women and men 



^^Overall, most sample members were black (46 percent) or Hispanic (43 percent). In six sites, 
more than two-thirds of the participants were black, while in three, more than two-thirds were Hispanic 
The 8 percent overall pioponion of white non-Hispanic sample members was disproportionately 
concentrated at the Phoenix Job Corps and, to a lesser extent, at Allentown in Buffalo, CET/San Jose, 
and EGOS in Denver. 

^^e proportion of males in each site ranged from a high of 61 percent at SER/Q)rpus Christi 
to a low of 36 percent at EGOS in Denver. In addition to SER/Corpus Christi, four other sites had 
male majorities: Chicago Commons, the East Los Angeles Skills Center, BSA in New York City, and 
CET/San Jose. 

AS- 



in the sample did vary among the sites, but the variation was much less than was the case for 
ethnic groups.^ 

Site differences were also large for several other subgroups of the research sample: 

• The amount of prior schooling varied among the sites more than gender 
and parenting status. Large differences in baseline educational attainment 
are important to bear in mind when examining rates of post-program GED 
attainment. Other factors aside, those who were closer to finishing high 
school at baseline are more likely to have received a GED at follow-up. 

• Employment during the year before random assignment varied among the 
sites even more than prior schooling. Holding all other observed factors 
constant, not having worked recently may signal either greater barriers to 
employment or more interest in schooling than in employment.^ 

• Public assistance receipt varied greatly. The percentage of a site's sample 
receiving public assistance may be a good indication of the relative income 
and job-readiness of the young people there.^ 

In subsequent chapters, program impacts for subgroups of the impact sample are 
presented. These subgroups are defined based on pre-random assignment (that is, pre- 
program) characteristics, and two types of analyses are presented.^ One approach splits the 



proportion of men acknowledging fatherhood ranged from almost none at the Los Angeles 
Job Corps, BSA in New York City, and the Atlanta Job Corps, to 18 percent at Chicago Commons. 
At CET/San Jose, the East Los Angeles Skills Center, and BSA in New York City, JOBSTART women 
were more likely not to be custodial mothers than to be living with their own children, while at 
Connelley in Pittsburgh, El Centro in Dallas, SEP/Corpus Christi, Chicago Commons, EGOS in Denver, 
the Phoenix Job Corps, and Allentown in Buffalo, the opposite was true. 

^More men than women worked in the year prior to random assignment, but for each group the 
proportion working also varied greatly among the sites. In most sites, the majority of women had noi 
worked in the year prior to random assignment, with the ratio of nonworking women to working women 
going above two-to-one at the Phoenix Job Corps. However, Connelley in Pittsburgh and CREC in 
Hartford were notable exceptions, with substantial majorities of women having had prior-year work 
experience in those sites. Men's prior employment profiles by site were quite the opposite, with ratios 
of employed to nonemployed as high as four-to-one at SER/Corpus Christi, CREC in Hartford, and the 
Atlanta Job Corps. Only among Los Angeles Job Corps men and men at BSA in New York City did 
nonworkers outnumber workers, with the former's ratio of nonworkers to workers exceeding two-to-one. 

^The average initial reading level on entering the program also varied among the sites. This is 
not dlsc'issed in detail because initial test scores are not available in all sites. 

^Detlning subgroups based on pre-random assignment characteristics is necessary in order to 
maintain tne legitimacy of comparisons of experimental and controls. For example, those who had not 
worked in the year prior to random assignment were just as likely to be randomly assigned to the 
experimental group as to the control group, making comparisons of experimental and controls with this 
characteristic appropriate. 
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entire sample into groups defined by a characteristic such as gender. Tnis "split file" subgroup 
analysis does not control for other measured differences among the groups, such as their site 
location or prior work experience. If a subgroup is concentrated in a few sites, as is the case 
for Hispanics, then the "split file" results may be reflecting site differences as much as subgroup 
differences. Since men and women are not concentrated in particular sites, the split file 
analysis presented for these groups in later chapters is appropriate. The second type of 
subgroup analysis presents results for designated subgroups that are statistically adjusted to 
account for other measured pre-program differences in the groups besides the characteristic 
used in defining them. For example, it presents results for ethnic subgroups controlling for 
differences in measured characteristics other than ethnicity. This analysis does control for site 
differences and thus can be used for subgroups that are relatively concentrated among a few 
sites. 

C. Sample Differences for Key Site Groupings 

In view of this report's special emphasis on sites' delivery systems for employment and 
training - whether concurrent, sequential/in-house, or sequential/brokered - Table 2.2 
collapses the 13 site columns of Table 2.1 into three columns, one for each type of delivery 
system.^^ Averaging data for sites in broad categories destroys much of the observed site 
variation - particularly regarding ethnicity, receipt of welfare and Food Stamps, parenting 
status, amount of schooling, and prior-year employment. 

Some of the observed site variation remains, however. Sample members at 
sequential/in-house sites were more likely to be custodial mothers, less likely to have 
acknowledged fatherhood on the enrollment form, less likely to have worked during the prior 
year, and more likely to have quit school during junior or senior year than were sample 
members in other sites. Sample members in concurrent sites we'-e older, more likely to have 
worked during the prior year, less likely never to have been married, less likely to be receiving 



^'The first eight columns of Table 2.1 are collapsed into the first column of Table 2.2, for 
concurrent sites. The next two columns of Table 2.1 - for El Centro in Dallas and the Los Angeles 
Job Corps - are collapsed into the *sequential/in-house* column of Table 2.2. Finally, the last three 
columns of Table 2.1 - for Allentown in Buffalo, BSA in New York City, and CREC in Hartford - 
are collapsed into the "sequential/brokered* colunm of Table 2.2. 
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TABLE 2.2 

CHARACTERISTICS AT THE TIME OF RANDOM ASSIGNMENT, BY PROGRAM STRUCTURE 





Sample 




Sequential/ Sequential/ 


A11 




Characteristic and Subgroups 


Size 


Concurrent 


In-House 


Brokered 


Cateaon' es 


D* 


Gender 














Women 


968 


50.3% 


57.4V 








Men 


871 


49.7 


42,6 


44.8 


d7 d 




Ethnicity 














White, non-Hispanic 


155 


9.8 


4.6 


a 1 




n nnn 
u . uuw 


Black, non-Hispanic 


840 


34.4 


DO • *f 


Do • 0 


d^i 7 




Hispanic 


783 


53 1 


29,2 


22 8 


AO ti 
. t) 




Other 


61 


2.7 


7.8 


. u 






Ethnicity, by gender 














Women 














White, non-Hispanic 


84 


5.1 


9 7 


A Q 


4.6*** 


0.000 


Black, non-Hispanic 


445 


18 7 




Id 7 


OA O 




Hispanic 


413 


25.5 


19.8 


15.3 


00 C 




Other 


26 


1.0 


3.8 


0.3 


1 4 




Men 














White, non-Hispanic 


71 


4.7 


1.9 


3.2 


3 9 




Black, non-Hispanic 


395 


15.7 


27.3 


33.8 


21 5 




Hispanic 


370 


27.6 


9.4 


7.5 


20. 1 




Other 


35 


1.7 


4.0 


0.3 


1.9 




Parental status 














Women living with 














own child{ren) 














No 


484 


25.4 


26.0 


29.5 


26.3*** 


0.005 


Yes 


484 


24.8 


31.4 


25 7 


26 3 




Men who have own child{ren) 














No 


765 


42.9 


40.5 


38.4 


41.6 




Yes 


106 


6.8 


2.1 


6.4 


5.8 




Employed within past year 














No 


870 


40.3 


62.7 


53.5 


47.3*** 


0 000 


Yes 


969 


59.7 


37.3 


46.5 


52.7 




Prior employment, by gender 














Women employed within 














podw jrCUi 














No 


547 


25.9 


39.1 


32.1 


29.7*** 


0.000 


Yes 


421 


24.4 


18.2 


23.1 


22,9 




Men employed within 














past year 














No 


323 


14.4 


23.6 


21.4 


17.6 




Yes 


548 


35.4 


19.0 


23.4 


29.8 




Sample size 


1,839 


1.120 


373 


346 
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TABLE 2.2 (continued) 



Samp^ e 



Sequential/ Sequential/ 



All 



Characteristic and Subgroups Size Concurrent In-House Brokered Categories p* 



Left school in grade 11 or 12 
No 
Yes 



1,078 
761 



59.9% 
40.1 



52.3% 
47.7 



61.3% 
38.7 



58.6%** 0.019 
41.4 



Received occupational 
training within past year 

No 

Yes 



1,529 
310 



81.1 
18.9 



90.6 
9.4 



81.8 
18.2 



83.1*** 0.000 
16.9 



Age 
16-19 
20 or 21 



1,359 
480 



71.3 
28.7 



80.2 
19.8 



75.4 
24.6 



73.9*** 0.003 
26.1 



Marital status 
Ever married 
Never married 



174 
1,665 



11.8 
88.2 



7.8 
92.2 



3.8 
96.2 



9.5*** 0.000 
90.5 



Living in own household or 
with boy/girlfriend 

No 

Yes 



1,500 
339 



80.9 
19.1 



88.7 
11.3 



76.0 
24.0 



81.6*** 0.000 
18.4 



Own AFDC case or receiving 
General Assistance 

No 

Yes 



1,344 
495 



75.1 
24.9 



74.3 
25.7 



65.3 
34.7 



73.1*** 
26.9 



o.noi 



Own AFOC case 
No 
Yes 



1,446 
393 



80.9 
19.1 



75.6 
24.4 



74.6 
25.4 



78.6** 
21.4 



0.012 



Receiving Food Stamps 
No 
Yes 



1,143 
696 



61.4 
38.6 



76.4 
23.6 



49.1 
50.9 



62.2*** 0.000 
37.8 



Arrested since age 16 
No 
Yes 



1,567 
272 



83.1 
16.9 



90.3 
9.7 



86.4 
13.6 



85.2*** 
14.8 



0.002 



Lived with both parents at 
age 14 
No 

Yes 

Sample size 



1,198 
641 



1,839 



58.7 
41.3 



1,120 



73.7 
26.3 



76.9 
23.1 



373 



346 



6E.1*** 0.000 
31.9 
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TABLE 2.2 (continued) 





Sampl e 




Sequential/ Sequential/ 


All 


Characteristic and Subgroups 


Size 


Concurrent 


In-House 


Brokered 


Categories p* 


Site 












Concurrent 












Atlanta Job Corps 


61 


5.4% 


0.0% 


0.0% 




CET/San Jose 


1S2 


13.6 


0.0 


0.0 


8.3 


Chicago Conmons 




6.6 


0.0 


0 0 


4 0 


Connel 1 ey (Pi ttsburgh) 


18' 


16.4 


0.0 


0.0 


10 0 


East LA Skil 1 s Center 


100 


8.9 


W . V 


0 0 




EGOS (Denver) 


183 


16.3 


0.0 


0 0 


10 0 


Phoenix Job Corps 


130 


11.6 


0.0 


0.0 


7 1 


SER/Corpus Christi 


236 


21.1 


0.0 


0.0 


12.8 


Sequential/in-house 












El Centro (Dallas) 


155 


0.0 


41.6 


0.0 


8.4 


LA Job Corps 


218 


0.0 


58.4 


0.0 


11.9 


Sequential /brokered 












Allentown (Buffalo) 


140 


0.0 


0.0 


40.5 


7.6 


BSA (NYC) 


119 


0.0 


0.0 


34.4 


6.5 


CREC (Hartford) 


87 


0.0 


0.0 


25.1 


4.7 


Sampls size 


1,839 


1,120 


373 


346 





SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all 1,839 sample members for whom there were 
24 months of follow-up survey data. Sample sizes reported may fall short of this number because of 
items missing from some sample members' questionnaires. 

Distributions may not total 100.0 percent because of rounding. 

^The column labeled "p" is the statistical significance level of differences among 
groups in distributions of characteristics: that is, p is the probability that observed proportions 
in each subgroup differ from one column to another only because of random error. A Pearson 
chi-square statistic was used to test the hypothesis of equal distributions. Statistical 
significance levels are indicated as = 1 percent; = 5 percent; * = 10 percent. 



AFDC, more likely to have lived vnth two parents at age 14, less likely to be black, and more 
likely to be Hispanic thai;, were sample members in other sites. 

D. Gender Differences in Baseline Characteristics 

An important question in evaluating JOBSTART is whether participation and program 
impacts varied by gender. A first step toward understanding these gender differences is to 
examine the other characteristics of the various gender-defined groups. Table 2.3 shows that 
men and women in the impact sample were similar in many characteristics, mcluding age, 
ethnic background, educational attainment, and initial reading levels. However, men were more 
likely to have had recent work experience and vocational training and to have been arrested 
fsince age 16. They were less likely to have been married, to be a parent, and to be receiving 
public assistance. 

Most of the differences between men and women just enumerated are due mainly to 
differences between mothers and other members of the sample. It is the mothers who are 
least likely to have worked in the year before random assignment, most likely to have lived on 
their own, most likely to have received AFDC and Food Stamps, and least likely to have lived 
with both parents at age 14. Because of these clear differences in initial characteristics 
between young mothers and other young women, the gender-based subgroup analysis that 
follows divides the female sample into these two groups. However, in most of the analysis, all 
men are grouped together because of the small number of men reporting that they were 
parents at baseline. 



2^ere may also be unobserved differences. For example, youths attracted to sequential/brokered 
programs run by community-based educational institutions may be more interested in passing the GED 
examination than are youths at concurrent sites run by training agencies. 

-52- 



TABLE 2.3 

CHARACTERISTICS AT THE TIHE OF RANDOM ASSIGNMENT, BY GENDER AND PARENTAL STATUS 



Women 



Men 







Living 




Do Not 










•J aiiii/ 1 c 


w{ f h Own 


A1 1 


nflVc uwn 


U ji u o Own 
nave i/ffii 


nl 1 




Characteristic and Subgroups 


Size 


Child(ren) 


Others 


Chlld(ren) 


Child(ren) 


Categories 




Ethnicity 
















White, non-Hispanic 


155 


7.2V 


10.1% 


9.0% 


1.9% 


8.4%*** 


0.000 


Black, non-Hispanic 


840 


54.1 


37.8 


43.5 


58.5 


45.7 




Hispanic 


783 


37.8 


47.5 


43.0 


38.7 


42.6 




Other 


61 


0.8 


4.5 


4.4 


0.9 


3.3 




Ethnicity, by gender 
















Women 
















White, non-Hispanic 


84 


7.2 


10.1 


0.0 


0.0 




0.000 


DiacKp non-nispamc 


A AC 


C A 1 

54 . 1 


n o 
3/ .0 


0.0 


0.0 


Z4.2 




Hispanic 


413 


37.8 


47.5 


0.0 


0.0 


22.5 




Other 


26 


0.8 


4.5 


0.0 


0.0 


1.4 




Men 
















Hni le 1 non-ni span ■ c 


7 1 

/ X 


U .U 


0.0 


9.0 


1 .9 


3.9 




Black, non-Hispanic 


395 


0.0 


0.0 


43.5 


58.5 


21.5 




Hispanic 


370 


0.0 


0.0 


43.0 


38.7 


20.1 




Other 


35 


0.0 


0.0 


4.4 


0.9 


1.9 




Employed within past year 
















No 


870 


62.4 


50.6 


38.2 


29.2 


47.3*** 


0.000 


Yes 


969 


37.6 


49.4 


61.8 


70.8 


52.7 




Prior employment, by gender 
















Women employed within 
















past year 
















No 


547 


62.4 


50.6 


0.0 


0.0 


29.7*** 


0.000 


Yes 


421 


37.6 


49.4 


0.0 


0.0 


22.9 




Men employed within 
















past year 
















No 


323 


0.0 


0.0 


38.2 


29.2 


17.6 




Yes 


548 


0.0 


0.0 


61.8 


70.8 


29.8 





ERIC 



Sample size 1,839 484 484 765 106 
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TABLE 2.3 (continuad) 



Characteristic and Subgroups 



Women 

Living 

Sample with Own A11 
Size Child(ren) Others 



Men 

Do Not 

Have Own Have Own A11 
Child(ren) Child(ren) Categories 



Left school in grade 11 or 12 
No 1,078 61.6% 58.1% 57.3% 57.5% 

Yes 761 38.4 41.9 42.7 42.5 



58.6% 0.487 
41.4 



Received occupational 
training within past year 

No 

Yes 



1.529 
310 



88.0 
12.0 



86.2 
13.8 



79.1 
20.9 



76.4 
23.6 



83.1*** 
16.9 



0.000 



Age 
16-19 
20 or 21 



1,359 
480 



61.6 
38.4 



85.1 
14.9 



77.9 
22.1 



50.0 
50.0 



73.9*** 0.000 
26.1 



I 

I 



Marital status 
Ever married 
Never married 



174 
1.665 



19.6 
80.4 



6.0 
94.0 



3.1 
96.9 



24.5 
75.5 



9.5*** 0.000 
90.5 



Living in own household or 
with boy/girlfriend 

No 

Yes 



1,500 
339 



55.8 
44.2 



86.8 
13.2 



94.9 
5.1 



79.2 
20.8 



81.6*** 0.000 
18.4 



Own AFDC case or receiving 
General Assistance 

No 

Yes 



1.344 
495 



37.6 
62.4 



83.9 
16.1 



87.8 
12.2 



79.2 
20.8 



73.1*** 0.000 
26.9 



Own AFDC case 
No 
Yes 



1,446 
393 



39.9 
60.1 



89.9 
10.1 



94.5 
5.5 



89.6 
10.4 



78.6*** 0.000 
21.4 
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Sample size 



1,839 



484 



484 



765 



106 
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TABLE 2.3 (continued) 



Women 



Men 



I 

I 







Living 





Do Not 








Sample 


with Own 


All 


Have Own 


Have Own 


All 


Lnaracien Stic and subgroups 


Size 


Child(ren) 


Others 


Child(ren) 


Child(ren) 


Categories 


Receiving Food Stamps 














No 


1,143 


41.1% 


67.6% 


72.8% 


56.6% 


62.2%*** 


Yes 


696 


58.9 


32.4 


27.2 


43.4 


37.8 


Arrested since age 16 














No 


1,567 


96.1 


93.8 


75.3 


67 9 




Yes 


111 


3.9 


6.2 


24.7 


32.1 


14.8 


Lived with both parents at 














age 14 














No 


1,198 


73.8 


64.7 


59.1 


71.7 


65.1*** 


Yes 


641 


26.2 


35.3 


40.9 


28.3 


34.9 


Site 














Concurrent 






• 








Atlanta Job Corps 


61 


3.7 


3.7 


3.1 


0.9 


3.3*** 1 


CET/San Jose 


152 


2.1 


13.4 


9.0 


7.5 


8.3 


Chicago Comnons 


74 


3.7 


2.9 


3.8 


12.3 


4.0 


Connelley (Pittsburgh) 


184 


13.0 


7.2 


8.9 


17.0 


IP.O 


East LA Skills Center 


100 


2.5 


6.6 


6.8 


3.8 


5.4 


EGOS (Denver) 


183 


13.8 


10.5 


8.0 


3.8 


10.0 


Phoenix Job Corps 


130 


7.6 


6.2 


7.3 


6.6 


7.1 


SER/Corpus Chnsti 


236 


11.0 


8.3 


15.9 


19.8 


12.8 


bequenii a i /i n-nouse 












El Centro (Dallas) 


155 


10.5 


6.8 


8.5 


5.7 


8.4 


LA Job Corps 


218 


13.6 


13.2 


11.2 


1.9 


11.9 


Sequential /brokered 












Allentown (Buffalo) 


140 


9.5 


7.4 


S.5 


15.1 


7.6 


BSA (NYC) 


119 


3.9 


7.0 


8.4 


1.9 


6.5 


CREC (Hartford) 


87 


5.0 


6.6 


3.S 


3.8 


4.7 



0.000 



Sample size 



1.839 



484 



484 



765 



105 



ERIC 
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TABLE 2.3 (continued) 



SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all 1,839 sample members for whom there were 24 months 
of follow-up survey data. Sample sizes reported may fall short of this number because of items missing from some 
sample members' questionnaires. 

Distributions may not total 100.0 percent because of rounding. 

^The column labeled ••p" is th« statistical significance level of differences aiTK>ng groups in 
distributions of characteristics: that is, p is the probability that observed proportions in each subgroup 
differ frm one column to another only because of random error. A Pearson chi -square statistic was used to test 
the hypothesis of equal distributions. Statistical significance levels are indicated as « 1 percent; = 5 
percent; * « 10 percent. 



CHAPTERS 



AN OVERVIEW OF JOBSTART SERVICES AND PARTICIPATION 

The JOBSTART model requires sites to opera basic education and occupational skills 
training classes that are interesting and accessible, effective in improving the sicills of young 
people, and of relatively long duration. It also requires young people to take advantage of 
these opportunities. Historically, education and training programs have had problems retaining 
young, economically disadvantaged dropouts (or even high school graduates).^ Thus, a key 
question for the evaluation is whether youths offered JOBSTART services do actually 
participate in lengthy, intensive services. 

This chapter looks at the JOBSTART experience from three perspectives.^' First, it 
briefly summarizes the nature of program services (highlighting key aspects of site variation) 
and reports youths' subjective reactions to the services.^ Second, it describes participation 
patterns of youths who were active in the JOBSTART Demonstration and compares that 
experience to other programs for young school dropouts. The analysis shows that participation 
was, in general, longer and more substantial than in most other JTPA Title IIA-funded 
activities for young dropouts, and that it was roughly comparable to participation in intensive 
programs such as the nonresidential Job Corps and the National Supported Work 
Demonstration (generally referred to simply as Supported Work). 

Third, the chapter analyzes the extent to which participation varied among different 
groups of youths and types of sites. This analysis finds that participation houi3 were similar 
for many groups: males and females, various ethnic groups, older and younger participants, 
youths with relatively higher and lower Ici^els of reading skills, ard recipients and nonrecipients 
of public assistance. Participation hours tended to be higher in labor markets with poorer 
employment opportunities. Average total participation hours were highest in sites that operated 



^U.S. Department of Education, 1988; Public/Private Ventures, 1988; Kelly, 1987. 

chapter summarizes and updates information in Chapters 2 and 4 through 8 of Auspos et al., 
1989. See that report for more details. 

^ese reactions were captured in the initial follow-up survey, which was conducted 12 months after 
random assignment, and in focus groups with partidpants. This section presents information on the 
JOBSTART components in specific sites. Chapter 6, which discusses program impacts by site, includes 
a summary of each site's program characteristics. 
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sequential programs with all services provided in-house (536 hours) followed by sites with 
concun^nt programs (405 hours) and sequential sites that referred participants to another 
agency for training (316 hours). Average hours in education were highest in sequential sites, 
while average training hours were highest at concurrent sites. 

The discussion of site experiences reinforces three basic themes of this report. FiKt, 
the variation in the details of the programs highlights the diversity of JOBSTART experiences 
among the sites within the general framework of the JOBSTART guidelines. Second, the 
experience of the sites shows that the basic program model can be implemented in a variety 
of administrative and labor market settings and using different basic program structures, though 
there were clearly stronger and weaker programs among the sites in the demonstration. Finally, 
this summary of how sites varied along many different dimensions provides background needed 
for interpreting the differences in site impacts that are presented in Chapter 6. 

I. The Nature of JOBSTART Services 

Basic education, occupational training, support services, and job placement assistance 
were available to participants in each site. To operate JOBSTART, two of the six community- 
based organizations (SER/Corpus Christi and Chicago Commons) added education to their 
regular service offerings, and three of the others (Allentown in Buffalo, BSA in New York 
City, and CREC in Hartford) developed or strengthened relationships with outside training 
programs so that they could serve as brokers, arranging training elsewhere for JOBSTART 
participants. The one community college (El Centro in Dallas) and three adult vocational 
schools (Connelley in Pittsburgh, the East Los Angeles Skills Center, and EGOS in Denver) 
had previously offered education and training but had to strengthen support services and job 
placement assistance. The three Job Corps Centers (in Atlanta, Los Angeles, and Phoenix) 
already had all four kinds of services in place. CET/San Jose already operated a program of 
integrated training and education, with support services and job placement assistance. 

Table 3.1 (which groups the sites by whether they operated concurrent, sequential/in- 
house, or sequential/brokered programs) describes the entry and exit ruJes, availability of 
separate classes for youths, expected duration of occupational training, and scheduled hours per 
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TABLE 3.1 

CHARACTERISTICS OF JOBSTART ACTIVITIES, BY SITE 





Fixed 




Expected 








Cycle or 


Separate 


Duration of 


Scheduled Hours per Day 






Open Entry 


Classes 


Occupat i ona I 




Site 


and Exit 


for Youths 


Training Education* 


Training Other Activities 


Total 



Concurrent 



Atlanta Job Corps 



I 

Ln 
I 



CET/San Jose 



Chicago Conmons 



Open entry 
and exit 



Yes 



Open entry 
and exit 



Fixed 
cycle 



In 

education 
only 

In 

education 
only 



1 year 

maximum^ 



600-1,000 
hours during 
23-3/ weeks 

500-1,380 
hours during 
22-42 weeks 



Individual ized, 
usually 2 hours 



2 hours, 
may vary 

J hours, 
i -b Jays 
per week 



Individual ized, 
usually 2.5 hours 
at start, more in 
subsequent weeks 



4.5 hours, 
my vary 



4.5-7 hours, 
depending 
on course 



Usually 2 hours 
in life skills 
and avocational 
activities at 
start, less in 
subsequent weeks^*^ 

None 



None 



6.5 hours 



6.5 hours 



6.5-8 hours 



Conneilcy (Pittsburgh) 



Fixed cycle 
with 

semesters 



Sometimes 
in education 



700-1,000 
hours 



2 hours 



4 hours 



1 hour of counsel' 
ing and other 
support services 
in school year 
1986-87^ 



6 hours in 
school year 

1985- 86, 

7 hours in 
school yei^r 

1986- 87 



East LA Skills Center 



EGOS (Denver) 



Open entry 
and exit 



Optn entry 
and exi t wi th 
semesters 



No 



In 

education 
only 



600-840 
hours during 
20-28 weeks 

600-1,000 
hours 



2 hours, 
may vary 



2 hours, 
may vary 



4 hours, 
may vary 



4 hours, 
may vary 



None 



None^ 



6 hours 



6 hours 



(continued) 
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TABLE 3.1 (continued) 



Site 



Expected 

Cycle or Separate Duration of 

Open Entry Classes Occupational 

and Exit for Youths Training Education* 



Scheduled Hours per Day 



Training 



Other Activities 



Total 



Phoenix Job Corps 



Open entry 
and exit 



Yes 



1 year 
tnaximum" 



Individualized, 
usually 2 hours 



Individualized, 
usually 2.5 hours 
at start, more in 
subsequent weeks 



Usually 2 hours 
in life skills 
and avocatlonal 
activities at 
start, less in 
subsequent weeks^*^ 



6.5 hours 



SER/Corpus Chriiti 



Fixed 
cycle 



Yes 



500-660 2.5 hours for 

hours during first 12-16 
22-23 weeks weeks* 



3,5 hours 
for first 12- 
weeks, then 
6 hours 



16 



Hone 



6 hours 



Sequential/ in-bcuse 
El Centro (Dallas) 

LA Job Corps 



Sequential/brokered 
Allentown (Buffalo) 



Open entry 
and exit 



Open entry 
and exit 



Open entry 
and exit 



In 

education 
only 



Yes 



In 

education 



for education, only 
varied in 
training 



720 hours 
during 24 
weeks 



1 year 

maximum" 



Varied by 

training 

provider 



3-4 hours 



3 hours for 
first 

10-12 weeks, 
then Individ- 
ual ized 



3 hours 



6 hours 



6 hours, 
may vary 



Varied by 

training 

provider 



2-3 hours in life 
skills activities 
during education 
phase^*^ 

3 hours in life 
skills or avoca- 
tlonal activities 
during education 
phase^*^ 



3 hours In life 
skills activities 
during education 



phase^ 



6 hours 



6 hours 



6 hours 
during 
education 
phase 



(continued) 



TABLE 3.1 (continued) 



Fixed Expected 

Cycle or Separate Duration of Scheduled Hours per Day 

Open Entry Classes Occupational 

Site and Exit for Youths Training Education* Training Other Activities Total 



BSA JiiYC; 


Open entry 
and exit 
for education, 
varied in 
training 


In 

education 
only 


Varied by 

training 

provider 


3 hours, 

4 days 
per week 


Varied by 

training 

provider 


3 hours In life 
skills activities 
during education 
phase, 4 days per 
week^ 


6 hours 

during 

education 

phase, 

4 days per 

week 


CREC (Hartford) 


Open entry 
and exit 
for education, 
varied in 
training 


No 


Varied by 

training 

provider 


3 hours 


Varied by 

training 

provider 


None^ 


3 hours 
during 
education 
phase 



SOURCE: Adapted from Auspos et a1., 1989. 



NOTES: ^Education hours refer to time spent in a basic education or GED-preparation class and do not include education provided as part of an 
occupational training course. 

'^Job Corps Centers offer a maximum of two years of training, but JOBSTART participants were supposed to be enrolled in courses that 
could be completed in one year. 

^Some participants worked in paid or unpaid work experience positions for limited periods. 

^Life skills classes typically provided instruction in work behaviors, goal setting, personal budgeting, health, and interpersonal 
relations. Avocational activities included physical education and driver education. 

^Additional hours were available on an individualized basis after the course ended. 



day in each site. In some sites, participants could enter courses at any time (open entry) 
and leave them when they had achieved a certain competency level (open exit), while in others, 
they had to adhere to a fixed cycle, with entry on specified dates and exit after a sec period 
of time. Some sites held classes for youths only, while others mixed youths and adults. Sites 
also varied in their expected duration of training, daily scheduling, and support services. 

A« Basic Education 

The education component typically consisted of individualized instruction, which allowed 
students to move at their own pace learning reading, mathematics, and other subjects needed 
to pass the GED examination. Mostly they worked on their own, doing workbook exercises 
or, less commonly, using computer-assisted instruction. In sites offering education and training 
concurrently, participants usually attended two hours of education classes and four hours of 
vocational training a day. In sites operating a sequential program, participants generally 
attended three hours a day of basic skills classes during the education phase, with the remaining 
three hours a day being devoted to life skills classes. 

The payment provisions of the contracts between service providers and funding agencies 
(especially local SDAs) were an important source of variation in the emphasis of the education 
component. In four sites (Connelley in Pittsburgh, EGOS in Denver, El Centro in Dallas, and 
SER/Co'7 , Chris ti), payment for education services was based on students passing the GED 
examination. This led these sites to make GED certification an important short-term goal of 
the program and to emphasize the skills tested on the GED examination in their education 
component. Other sites - CET/San Jose, the East hos Angeles Skills Center, and especially 
Chicago Commons - saw GED attainment as a long-term goal and did not stress it in their 
JOBSTART programs, focusing more on improving basic skills as an aid to vocational training 
and job placement. 

The actual curricula and instructional materials were not specified by the JOBSTART 
guidelines. The three Job Corps sites used the standard Job Corps materials (workbooks, 
textbooks, and audiovisual materials), though two centers (Atlanta and Phoenix) also had 
supplementary computer-assisted instruction. The three sequential/brokered sites used the 
Comprehensive Competencies Program (CCP) developed by U.S. Basic Skills Investment 



"^is grouping was chosen because, as discussed later, participation rat&s by component, participation 
hours, and program emphasis differed among these three types of sites. 
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Corporation. CCP is an instructional management system integrating textbooks, workbooks, 
computer software, audiovisual materials, and progress tests. In the seven other sites, teachers 
developed their own instructional mateuals using a variety of sources, such as GED preparation 
courses and reading and mathematics textbooks that used the "mastery learning" approach, 
which focuses on the step-by-step acquisition of specific competencies. In four of these sites, 
staff supplemented pencil and paper exercises with computer-assisted instruction.^ 

Teachers in most sites felt that the individualized, self-paced instruction provided a better 
learning environment than participants had typically found in high school. The competency- 
based cc jrses allowed the youths to see themselves making incremental progress as they 
advanced toward what was, for many, a remote goal of mastering basic skills and receiving a 
GED. Most students preferred this instructional approach because they felt that it made them 
active participants in the process of learning and allowed them to master one topic before 
beginning another. In the follow-up survey, about three-fourths of JOBSTART participants 
found self-paced instruction "very helpful," while virtually no one found it "not helpful at all." 

Yet students also valued interaction with instructors, as much for the personal attention 
and motivation it provided as for instruction in specific skills. About 75 percent of JOBSTART 
participants rated support from teachers and fellow students in the education component "very 
helpful." 

Despite the overall favorable assessment, three concerns emerged. First, with a few 
important exceptions, the basic education and skills training activities operated separately, with 
little integration of material. As discussed in Chapter 1, only at CET/San Jose, and to a lesser 
extent Chicago Commons, were basic skills and occupational training instruction truly 
integrated. Though several other sites did attempt to coordinate the two activities to a limited 
extent (creating a distribution of sites rather than two clear-cut categories), these sites fell short 
of the integration observed at CET/ San Jose and Chicago Commons. Second, some instructors 
thought the curriculum should include more material on critical thinking and general 
knowledge, in contrast to the functional literacy and mathematics emphasis of many integrated 
programs. Third, some instructors said that students with very poor skills or low motivation 
found the work boring and, as a remedy, suggested more group activities. One site. El Centro 



Sec Auspos el al., 1989, for the details of these programs. 

-63- 



in Dallas, shifted to this approach, relying more heavily than other sites on class exercises and 
lectures. 

B. Occupational Skills TrainiM 

The choices of occupational training available to participants varied among the sites. 
Participants at large vocational schools could choose courses in more than 20 occupational 
areas. The Job Corps Centers and larger community-based organizations (CBOs) also offered 
a wide range of vocational training. In contrast, SER/Corpus Christi, which provided training 
in-house, offered only a few courses. 

In theory, youths in sequential/brokered sites could choose courses from a variety of 
local agencies. However, in practice, some courses were unavailable to them because they 
could not satisfy entrance requirements, or other difficulties prevented them from gaining 
entry.^ As discussed later in this chapter, the resulting low rate of participation in training in 
sequential/brokered sites was the major operational issue concerning the training component. 

As a group, JOBSTART participants were enrolled in training for a broad range of 
occupations - clerical and service jobs, machine trades, benchwork occupations, and structural 
work such as welding. Occupational choices for men and women followed traditional patterns, 
as shown in Tabl ' with aboui three-iourths of the women participants training for clerical 
jobs.^ 

Using categories employed by the U.S. General Accounting Office (GAO) in a recent 
analysis of JTPA Title HA adult training, MDRC classified the JOBSTART training provided 
to participants as leading to jobs requiring low or low/moderate skills (slightly less than one- 
fourth of participants), moderate skills (about one-half of participants), and higher skills (about 
one fourth of participants),^ This distribution of skills ratings for training occupations was 
similar to what the GAO found for JTPA Title HA adult programs. This was unexpected, 



^As mentioned in Chapter 1, the JTPA performance standards created an incentive for SDAs to 
emphasize lower-cost, short-term programs. Some SDAs in study sites were reluctant to provide a single 
individual with both education and training, and many JTPA Title IIA-funded service providers operated 
under performance-based contracts linking payment to placement in a job. Both practices hindered the 
efforts of JOBSTART youths in sequentialA)rokered sites to find a training agency willing to accept them. 

^This table, taken from an earlier report on JOBSTART (Auspos et al., 1989), is based on a similar, 
but slightly smaller, sample than that used for this report. 

^See U.S. General Accounting Office, 1988, for the definitions of categories of training. The 
percentage distribution reported in the text for JOBSTART was calculated in Auspos et al., 1989» based 
on a slightly different sample of participants than that used in this report. 
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TABLE 3.2 



PERCENTAGE DISTRIBUTION OF OCCUPATIONS 
FOR PARTICIPANTS IN TRAINING, BY GtNDER 



Training Category^ 


Men 


Women 


Total 


Cl€f*ical and sales occuDation' 








Stenography, typing, filing. 








and related occupations 


5.0% 


51.3% 


29.2% 


Computing and account-recording 


7.0 


20.3 


13.9 


Production and stock derks. 








and related occupations 


0.3 


0.0 


0 1 


Information and message distribution 


1.4 


0.0 


0 7 


Miscellaneous clerical 


0.0 


0.5 


0.3 


Sales and consumable conmodi ties 


0.8 


1.0 


0.9 


Total 


14.6 


71 1 

/J.J 




Service occupations 








Food and beverage preparation and services 


3.1 


4 6 


3 9 


Miscellaneous oersonal services 


0.6 


11 5 


6 1 

U.J 


Building and related services 


8.4 


2.1 


5.1 


Total 


12.0 


18.2 


15.3 


Machine trades occupations 








Metal machining 


5.0 


0.8 


2 8 


Mechanics and machinery repair 


22.4 


1.3 


11-4 


Printi ng 


0.6 


1.3 


0.9 


Wood machining 


0.6 


0.5 


0 5 


Total 


28.6 


3.8 


15.7 


Benchwork occupations 








Assembly and repair of electrical equipment 


11.5 


1.3 


6.2 


Painting, decorating, and related occupations 


0.8 


0.3 


0.5 


Fabrication and repair of plasties, synthetics. 








rubber, and related products 


2.2 


0*3 


1.2 


Fabrication and repair of textile, leather, 








and related products 


1.7 


0.3 


0.9 


Total 


16.2 


2.1 


8.8 


Structural work occupations 








Metal fabricating 


9.8 


0.5 


5.0 


Welders^ cutters, and related occupations 


0.8 


0.0 


0.4 


Electrical assembling, installing, and repairing 


5.9 


0.5 


3.1 


Painting, plastering, waterproofing. 








cementing, and related occupations 


1.7 


0.0 


0.8 


Construction 


8.4 


1.3 


4,7 


Total 


26-6 


2.3 


13.9 


Miscellaneous occupations 








Transportation 


0.0 


Oe3 


0.1 


Graphic art work 


2.0 


0.3 


1.1 


Total 


2.0 


0.5 


1.2 


All training categories 


100.0 


100.0 


100.0 


Number of participants in training 


357 


390 


747 



(continued) 
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TABLE 3.2 (continued) 



SOURCE: Adapted from Auspos et al . , 1989. The categorization of occupations is 
derived from U.S. Department of Labor, 1977. 

NOTES: Calculations for this table used data for all youths who were active for at 
least one hour in a JOBSTART training component within 12 months of random assignment and 
responded to the 12-month follow-up survey. 

Distributions may not total 100.0 percent because of rounding. 

Individual category totals may not equal the general category totals because 

of rounding. 

Tests of statistical significance were not run. 

^Individuals participating in more than one training category are included 
In the category in which they attended the most hours. 
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since JOBSTART participants faced more barriers to employ^ient than did the typical JTPA 
Title IIA adult client. 

One argument for sequential programs is that the up-front education allows participants 
to enter more advanced training. In the JOBSTART Demonstration, this did not appear to 
occur. In terms of the same GAO categories, the jobs that youths trained for in sequential 
sites did not appear to require higher skills than those in concurrent sites. 

C. Sup port Seniles To Facilif «te ParticiDation 

All sites pro%aded basic support services such as assistance with transportation and child 
care, which helped participants attend the program, as shown in Table 3.3. All sites provided 
bus passes or small allowances to cover the coste of commuting to the program. JOBSTART 
counselor/coordinators placed a high priority on adequate child care arrangements. In most 
sites, staff referr.-.d JOBSTART participants to other agencies for child care, with the expenses 
being covered by JTPA or the Work Incentive (WIN) program. The Atlanta Job Corps, two 
CBOs (SER/Corpus Chrisli and CET/San Jose), and one adult school (Connelley in Pittsburgh) 
had on-site day care facilities, but staff reported that students frequently preferred to make 
their own arrangements in their own neighborhoods. 

Many sites also found ways to provide small payments to meet other costs of 
participating, though the Job Corps Centers were consistently able to provide more support 
than the other sites. Ten of the 13 sites were able to provide some type of small needs-based 
payment, while nine of the sites provided on-site meals or food to take home or special 
allowances for clothing or to meet rent emergencies. Seven of the sites (including all three 
Job Corps Centers) provided some form of incentive payments to participants who reached 
milestones in the program. The Job Corps Centers also provided on-site medical and dental 



care. 



In addition, to increase participants' motivation and commitment to the program, site 
staff used a variety of strategies: personal counseling, peer support, time management training, 
and group recreational activities. Finally, staff at most sites provided training in life skills - 
covering topi^ .uch as health, personal finances, and workplace routines - to help the young 
people function more responsibly and productively in a variety of roles and situations. Six ol 
the sites (the three Job Corps programs. El Centro in Dallas. Allentown in Buffalo, and BSA 
in Nev/ York City) incorporated two to three hours of formal life skills classes into the regular 
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TABLE 3.3 

BASIC SUPPORT SERVICES AVAILABLE IN JOBSTART, BY SITE 



SUe 



Needs-Based Payments Transportation 



Child Care 



Other 



Job Corps Centers 
Atlanta Job Corps 



LA Job Corps 



I 

ON 

CO 
I 



Phoenix Job Corps 



Basic allowance of 
$40 per month for 
first 2 months, $60 
for next 3 months, 
$80 after 5 months 



Basic allowance of 
$40 per month for 
first Z months, $60 
for next 3 months, 
$80 after 5 months 



Basic allowa.^e of 
$40 per month for 
first 2 irtonths, $60 
for next 3 months, 
$80 after 5 months 



Bus passes 



On-site 



Bus passes 



By referral 



Bus passes 



By referral 



Incentive Payments 



Free mealsj clothing 
allowance of $75 in 
first month, $50 in 
third month, $96 in 
sixth and tenth months, 
$51 in twelfth month; 
on-site medical and 
dental care 



Free meals; clothing 
allowance of $75 in 
first month, $50 in 
third month, $96 in 
sixth and tenth months, 
$51 in twelfth month; 
on-site medical and 
dental care 



Free meals; clothing 
allowance of $75 in 
f i rst month, $50 in 
third month, $96 in 
sixth and tenth months, 
$51 in twelfth month; 
on-site medical and 
dental care 



Schools 



Merit raises can increase 
basic allowance to $100 per 
month after 6 months; $75 
per month is placed in 
escrow for enrol lees who stay 
6 months, which increases 
to $100 per month after 6 
months; $150 bonus in tenth 
month 

Merit raises can increase 
basic allowance to $100 per 
month after 6 months; $75 
per month is placed in 
escrow for enrol lees who stay 
6 months, which increases 
to $100 per month after 6 
months; $150 bonus in tenth 
month 

Merit raises can increase 
basic allowance to $100 per 
month after 6 months; $75 
per month is placed in 
escrow for enrol lees who stay 
6 months, which increases 
to $100 pt*^ month after 6 
months; $150 bonus in tenth 
month 



Connel 1 ey 
(Pittsburgh) 



1 1 ' 



$5 per day^ 



$2 per day or bus passes^ On-site and by 

referral 



$50 one-time clothing 
grant 



$50 for passing 6ED;^ 
$50 for each month of perfect 
attendance; quarterly paiyment 
of $50 for "A" average, $25 
for "B** average, $10 for •'C** 
average | ^ ^ 
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TABLE 3.3 (continued) 



Site Needs-Based Payments Transportation Child Care Other Incentive Payments 



East LA 
Skills Center 


None 


Bus passes, gasol ine 
vouchers 


By referral 


Emergency funds, 
lunch money .during 
a brief period 


None 


EGOS (Denver) 


None 


Bus passesp gasoline 
vouchers 


By referral 


Lunch nwney 

during a brief period 


None 


El Centre (Dallas) 

Comunity-based 
organizations 


$5 per day 


Bus passes 


By referral 


Emergency rent funds 


$5 per week for perfect 
attendance 


Al 1 entown 
(Buffalo) 


$1 per hour if on AFDC, 
otherwise $2 per hour, 
during education and 
training 


Included in needs- 
based payment 


By referral 


None 


None 


BSA (NYC) 


$23-$30 per week during 
education,^ $30 per week 
during JTPA training 


Included in needs- 
based payment; tokens 
available otherwise 


By referral , 
$15 per week 
for expenses 


Free breakfasts 


$5 for weekly academic 
progress; $5 for perfect weekly 
attendance" 


CET/San Jose 


$1 per hour, for farm- 
workers only 


Bus passes for farm- 
workers and others who 
demonstrate need 


On-site and by 
referral 


Weekly food bank to 
provide free 
groceries 


None 


Chicago Commons 


$6 per <iay 


Included in needs- 
based payment 


By referral 


None 


None 


CREC 

(Hartford) 


None 


Bus passes 


By referral 


None 


None 


SER/Corpus Christi 


$8 per day 


Included in needs- 
based payment 


On-site for 
children over 18 
months and by 
referral 


None 


$20 for each grade-level gain 
in reading; $20 for passing GED 
pre-test; $40 for passing GED 
test; $45 for "A** average 
throughout occupational training, 
$25 for "B" average 



ERIC 



SOURCE: 
NOTES: 



i:s 



Adapted from Auspos et al 



1989. 



*At intervals, this site combined transportation and needs-based payments into one $7 per day payment. 
"Available during 1986-87 school year. 
^During October 1986-August 1987. 
^Available after October 1987. 
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program day. The remaining seven sites did not focus as systematically on life skills, instead 
incorporating these topics into the training curriculum, counseling or group discussion sessions, 
or occasional lectures. 

Youths cited personal attention from staff as a crucial aid in helping them move toward 
self-sufficiency. While agencies that traditionally served disadvantaged youths typically offered 
these support sendees from the beginning of the demonstration, a number of sites accustomed 
to serving adults increased this type of activity as their programs evolved. 

Clearly, the support services and other activities available at the Job Corps Centers 
surpassed those at the other sites in both breadth and inten ity. To a large extent, the 
JOBSTART participants in these sites were able to partake of the full array of Job Corps 
activities, including recreational, health, and food services. However, AJlentown in Buffalo, 
Connelley in Pittsburgh, and SER/Corpus Christi also offered high levels of these services. 

D, Job Placement Services 

Sites were required to assist youths in finding training-related employment, but this phase 
of the program typically received less attention than others. Nearly all the sites did provide 
instruction about employers* expectations as well as job search techniques. About one-half of 
the sites arranged paid or unpaid part-time work experience positions for some participants 
during the program. Approximately one-fourth of a sample of participants worked at some 
point - in program-arranged or self-initiated jobs - while they were active in the program. 
Those in the sample who were employed worked an average of about 50 percent of the weeks 
they were in the JOBSTART program and were employed an average of about 30 hours per 
week during the weeks they worked- During the months they worked, their hours of classes 
in JOBSTART were lower than were those of nonworking participants. 

Efforts to find participants permanent employment typically began near the end of 
training, with instructor contacts serving as an important source of information about job 
openings. Since many youths left the program without reaching this stage, it is not surprising 
that only about one-fourth of participants reported that program staff referred them to a job 
or told them about openings. 



life skills curricula in these sites was oriented around daily living, with units on health education, 
substance abuse prevention, sexuality and family planning, personal finances, civics, communication skills, 
goal-setting and planning, and improving one's self-esteem. 
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Job placement assistance was especially strong at CET/San Jose, Chicago Commons, and 
the Job Corps program in Phoenix. In all these sites, instruction in proper work behavior, 
employer expectations, and job search techniques began while students were still in training; 
placement specialists provided leads and assistance in finding a job; and CET/San Jose and 
Chicago Commons had especially strong ties to local employers. 

Job placement assistance was noticeably weaker at Allentown in Buffalo, BSA in New 
York City, and CREC in Hartford (the three sequential^rokered sites), and at the Atlanta Job 
Corps, the East Los Angeles Skills Center, EGOS in Denver, and SER/Corpus Christi (among 
the concurrent sites). At the first three of these sites, job placement was intended to be the 
responsibility of the training agency, but most participants never were active in that component 
so only informal assistance was available from the JOBSTART agency. Those concurrent sites 
with weak job placement typically lacked any or sufficient job development specialists on staff, 
were larger agencies with no special emphasis on placing JOBSTART youths, or contracted out 
job placement to another organization that did not see the JOBSTART youths as a high- 
priority croup. 

E. Scheduling. Daily Service Mix, and Planned Program Duration 

Sites also varied in the way they scheduled classes and the expected duration of their 
programs. The demonstration sites scheduled JOBSTART classes in three basic ways. The 
majority of sites that operated both the education and training components themselves 
scheduled the classes on an "open-entry/open-exit" basis. This means that participants could 
enter the program at any time, progress through the material at their own pace, and complete 
the course whenever they reached the specified competency levels. The duration of training 
was open-ended, but sites anticipated that participants would typically be able to complete the 
prescribed training curriculum in many fields in approximately 600 to 800 hours. Individuals 
who needed additional time to complete competencies could stay longer, however. 

In a second program variation, some concurrent sites operated JOBSTART as a series 
of "fixed cycles," meaning that all participants started and completed training together on 



^^ese sites included concurrent sites (the Atlanta Job Corps, CET/San Jose, the East Los Angeles 
Skills Center, and the Phoenix Jobs Corps) and sequential/in-house sites (El Centro in Dalla.s and the Los 
Angeles Job Corps). 
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specified dates and the maximum length of training was prescribed." In a third variation, the 
three sequential/brokered sites operated the education component on an open-entry/open-exit 
schedule, but the training schedule was determined by the variety of training organizations at 
which JOBSTART participants were enrolled. 

Sites also showed great variety in the number of hours scheduled for activities each day. 
The usual schedule ranged from a low of three hours per day at CREC in Hartford to seven 
to eight hours per day in some courses at Chicago Commons. A typical day can be described 
in terms of three basic models: 

• Concurrent sites that were CBOs or schools. Students typically had six 
hours of classes per day, five days a week. In general, two hours were 
spent in education classes, with training classes scheduled for the 
remaining four hours. 

• Concurrent sites that were Job Corps Centers. These sites had six and 
a half class hours per day. Schedules were highly individualized and 
changed frequently, but commonly included two hours of education, two 
and a half hours of vocational training, and two hours devoted to life 
skills, health education, or avocationa! activities such as sports. 

• Sequential sites. These also scheduled a six-hour day during the 
education phase, but the daily distribution of activities was quite 
different. Typically, three hours were spent in education classes and 
another three hours were spent in life skills training. The training 
schedules were set by the training providers at the brokered sites, but 
typically involved five to six hours of classes per day. Training classes 
ran for six hours a day at the sequential/in-house sites. 

The duration of the occupational training component also varied among the sites, 
ranging from 22 to 23 weeks at SER/Corpus Christi to a year at the Job Corps sites. Even 
within a site, there could be significant variation among the different training options. At 
Chicago Commons, for example, scheduled training ranged from 500 hours in industrial 
inspection to 1,380 hours in packaging-machine repair. 

This diversity in scheduled daily hours and program duration meant that the planned 
participatio i hours for youths varied greatly across the sites, with the greatest variation showing 
in the training component. At SER/Corpus Christi, a participant completing education and 



"These sites included Chicago Commons, Connelley in Pittsburgh, EGOS in Denver, and SER/Corpus 
Christi. 
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training in about six months, as planned, would have had no more than 660 hours of 
occupational training. In contrast, one training course at Chicago Commons totaled nearly 
1,400 hours, and a sequential program such as the one operated by the Los Angeles Job Corps 
could last for as long as a year.^*^ 

F. S ummary of Program Implementation by Site 

Table 3.4 rates the implementation of the four central JOBSTART components in each 
site. (See Appendix Table D.l for the details behind these ratings.) The information in Table 
3.4 and the material already presented in this section suggest that the four key components of 
the JOBSTi\RT program were implemented most successfully at CET/San Jose,^^ Chicago 
Commons, Cbnnelley in Pittsburgh, El Centro in Dallas, the Los Angeles Job Corps, and the 
Phoenix Job Corps and least successfully at the Atlanta Job Corps, BSA in New York City, 
CREC in Hartford, and EGOS in Denver To summarize the ratings by component: 

• Education. Most sites that chose to operate a separate education 
component were able to offer an activity meeting the JOBSTART 
guidelines. The two sites with noticeably weak education activities were 
the Atlanta Job Corps (where unclear objectives for education and staff 
turnover hampered implementation) and CREC in Hartford (where 
computer facilities were under-utilized and poor attendance was a serious 
problem). 

• Training. The training component showed the most variation - 
primarily because in sequential/brokered sites most youths never 
participated in training. In addition, the limited training offerings, less 
experienced staff, and older equipment of SER/Corpus Christi (reflecting 
the common problems of a community-based organization) hampered 
its ability to implement the training component. 

• Support services. Although the Job Corps Centers did offer substantially 
more services than other sites, all programs were able to provide the 
limited types of support services called for in the JOBSTART guidelines: 
assistance in arranging and/or financing child care and transportation to 
and from coursework. Allentown in Buffalo, Connelley in Pittsburgh, 
and SER/Corpus Christi, in addition to the three Job Corps Centers, 
provided a noticeably longer list of services, including better needs-based 



^^Job Corps Centers offer a maximum of two years of training, but JOBSTART participants were only 
to be enrolled in courses that could be cx)mpleted in one year. 

^^CET/San Jose provided most of its basic education services within training activities, so this overall 
characterization reflects a judgment about the other three components and the way in which education was 
incorporated into the training component. 



-73- 



TABLE 3,4 



RATINGS OF THE IMPLEMENTATION OF JOBSTART COMPONENTS. BY SITE 



Support Job 
Site Education Training Services Placement 



Concurrent 



Atlanta Job Corps Medium 

CET/San Jose No rating 

Chicago Commons Medium 

Connelley (Pittsburgh) High 

East LA Skills Center Medium 

EGOS (Denver) Medium 

Phoenix Job Corps Medium 

SER/Corpus Christi High 

Sequential I in-house 

El Centro (Dallas) High 

LA Job Corps Medium 

Sequen tial /broke red 

Allentown (Buffalo) High 

BSA (NYC) Medium 

CREC (Hartford) Low 



Medium High Low 

High Medium High 

High Medium High 

High High Medium 

Medium Medium Low 

Medium Medium Low 

High High High 

Low High Low 

Medium Medium Medium 

Medium High Medium 

Low High Low 

Low Medium Low 

Low Medium Low 



SOURCE: MDRC operations staff. 

NOTES: See Appendix Table D,l for details of the implementation of components in 
each site. 

^In this site, a separate rating of the education co^nponent was 
inappropriate because education and training were more integrated than in other sites and 
staff strongly emphasized training over passing the GED examination. 
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payments, life skills training and counseling, and a method of identifying 
service needs and making referrals of youths to other agencies providing 
the required services. 

• Job placement assistance. Most sites offered job placement assistance 
that fell short of the JOBSTART guidelines, either because many youths 
never received the service (especially in sequential/brokered sites) or too 
few staff with a specialty in job search assistance were assigned to work 
with the JOBSTART youths. As mentioned earlier, job placement 
assistance was especially strong at CET/San Jose, Chicago Commons, and 
the Phoenix Job Corps. 



II. The Intensity of JOBSTART Participation 

Partic'pation in JOBSTART among experimentals was measured by participation rates 
in each activity, hours of participation in each activity, and overall length of stay. Table 3.5 
shows these summary measures for all experimentals in the impact sample: 

• Participation rates. Nearly 90 percent of all experimentals in the impact 
sample participated in JOBSTART to some extent. Eighty-six percent 
of all experimentals (and nearly all of those who were active in 
JOBSTART) attended basic skills education classes, while 67 percent 
participated in training, and 40 percent participated in other activities, 
which were optional for sites. 

• Participation hours. Average hours were 128 in education, 249 in 
training, and 37 in other activities, for a total of 415 hours. Forty 
percent of all experimentals spent feyer than 201 hours in all 
JOBSTART activities; 26 percent spent 201 to 500 hours, and 35 percxjnt 
spent more than 500 hours. 

• Length of stay. The average length of stay was 6.8 months, with the 
median length being slightly less, 6 months; 78 percent of experimentals 
were active for 3 months or more, while 54 percent stayed in the 
program for 6 months or more. This was measured from the time of 
random assignment through the last month that included any hours of 
participation.^^ Sixteen percent of the experimental sample were still 



^'*These averages and those cited in the next paragraph include the 11 percent of the sample with zero 
hours of JOBSTART activities. 

^^The period of participation could include months of inactivity if a person stopped attending classes 
and then returned to the program within the 12-month follow-up period. However, this does not appear 
to have been a common pattern. Among a sample of participants, about 85 percent did not have any 
months of inactivity within the period they were counted as active, and among those with inactivity, the 
average period of inactivity was about two months. Youths who attended JOBSTART were counted as 

(continued...) 
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TABLE 3.5 



PARTICIPATION RATES, HOURS OF PARTICIPATION, AND LENGTH OF STAY 

FOR EXPERIMENTALS 



Activi ty Measure 



Experimental s 



Percent participating in 
Any activity 
Education 
Trai ning 

Education and training 
Other activities 



88.7 
85.9 
66.6 
64.4 
40.0 



Average hours in 
Education 
Trai ning 

Education and training 
Other activities 
All activities 



128.1 
248.9 
377.0 
37.3 
414.8 



Percentage distribution of hours 
in education and training 
None 

Up to 200 
201 to 500 
501 to 700 
701 or more 
Total 



11.9 
33.2 
22.4 
15.5 
17.0 
100.0 



Percentage distribution of hours 
in all activities 
None 

Up to 200 
201 to 500 
501 to 700 
701 or more 
Total 



11.3 
28.6 
25.5 
15.2 
19.5 
100.1 



Length of stay (months) 
Average 
Medi an 



6.8 

6.0 



Percent still participating in month 
3 
6 
9 

12 
15 
18 

19 or later 



78.0 
53.6 
30.6 
16.4 
9.6 
4.8 
3.7 



Number of experimental 



949 



(continued) 



TABLE 3*5 (continued) 



SOURCE: MDRC calculations from JOBSTART enrollment form, MIS, and survey 

data« 

NOTES: Calculations for this table used data for all experimental for 
whom there were 24 months of follow-up survey data, including those with values 
of zero for outcomes and those who were assigned to JOBSTART but did not 
participate* 

Distributions may not total 100.0 percent because of rounding. 



active in the program in the twelfth month after random assignment, 
while 10 percent were still active in the fifteenth month. 

These findings shoM' that JOBSTART succeeded in engaging more than half of the 
youths in the experimental group in the program and its activities, but that for about 40 
percent of them, participation was quite low and JOBSTART did not constitute an intensive 
or lengthy program. Because of this wide range of participation levels, v/ith some people 
having very few hours, the average total hours for the sample as a whole is the equivalent of 
slightly less than three and one-half months of regular attendance for six hours per day. Most 
people in the sample did not participate long enough to get a GED or complete a training 
course. 

To place these results in context, JOBSTART participation may be compared to 
reported participption in other programs for young, disadvantaged school dropouts. Length of 
participation is a simple measure that permits comparisons with three types of youth programs: 
JTPA Title IIA programs for young dropouts, the Job Corps, and Supported Work.^^ JTPA 
Title IIA typically provides relatively short-term activities, while the Job Corps and Supported 
Work have been among the most intensive employment and training programs for 
disadvantaged youths. In these comparisons, either the average or median length of 
participation is used, depending on the available data. 

Overall, JOBSTART participants stayed in the program considerably longer than did 
young dropouts in JTPA Title IIA activities, as shown in Table 3.6. During program year 
1986, when the demonstration was in operation, the median length of participation for all 
young dropouts in JTPA Title IIA programs was 3.4 m^:siV^ comparf,d to 6 months for 
JOBSTART.^^ JOBSTARTs median length of participation exceeded that of young dropouts 



^^(...continued) 

participating for the entire month in which they were randomly assigned and all months in which they 
showed any JOBSTART hour<. The measure might have overestimated the length of participation 
somewhat when a youth was randomly assigned late in a month or ended participation early in a month. 

^^For information on the Job Corps, see Richardson and Burghardt, 1985, and U.S. Department of 
Labor, 1987. On Supported Work, see Maynard, 1980. For JTPA Title IIA, see U.S. Department of 
Labor, 1988. Hours of attendance are not reported for all programs, hence this comparison uses length 
of stay, for which the data are available. 

^'The average length of participation in JTPA Title IIA programs is not available from published 
sources. The figure for JTPA Title IIA includes only persons who actually participated, while the 
JOBSTART figure includes the 11 percent of the sample made up of nonparticipants with ze.u months 
of activity. 
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TABLE 3.6 



PARTICIPATION AND LENGTH OF STAY FOR YOUNG DROPOUTS 
IN JTPA TITLE IIA, BY ACTIVITY 



Activity 


Percentage 
Distribution of 
Youths in JTPA 


Median Length 
of Stay 
(Months) 


Classroom activities 






Basic education 


22.8 


3.71 


Occupational skills training 


15.6 


3.98 


Combined basic education and 






occupational skills training^ 


4.6 


6.97 


Total 


42.9 


3.97 


On-the-job training 


12.2 


3.14 


Job search assistance 


15.3 


0.81 


Work experience 


7.8 


3.67 


Other services 


21.8 


3.59 


Any activity 


100.0 


3.40 



SOURCE: U.S. Department of Labor. Division of Performance Management and 
Evaluation, 1988. 



NOTES: This table includes data for young dropouts served under JTPA Title 
IIA during program year 1986, 

^JTPA data (as recorded by the U.S» Department of Labor. Division 
of Performance Management and Evaluation. 1988) combined basic education and 
occupational skills training under the label CT-Other» 



in all JTPA components except one. The exception was a program combining basic education 
and occupational skills training, a mix similar to JOBSTARl 's, which had a median length of 
7 months but was offered to only 5 percent of all young dropouts in JTPA Title IIA activities. 
For JOBSTART participants active in both education and skills training, the median length of 
slay in the program was also approximately 7 months. These findings support the conclusion 
that JOBSTART achieved its goal of operating a program more intensive than that typically 
offered in JTPA Title IIA programs for young dropouts. 

JOBSTARTs average length of participation was similar to those of the Job Corps and 
Supported Work. During program year 1986, the average stay in the Job Corps was 6.9 
months, compared to JOBSTARTs average of 6.8 months.^^ Supported Work was an 
experimental program of paid work experience under conditions of gradually increasing 
responsibility on the job, close supervision, and work in association with a crew of peers. It 
operated from 1975 to 1979 and included young school dropouts, many with a criminal record, 
as one of its target groups. While precise comparisons are impossible, the length of 
participation in the two programs appears to have been similar. The average length of 
participation in Supported Work was 6.7 months (close to that in JOBSTART) and the median 
was approximately 6 months (the same as in JOBSTART), but 25 percent of Supported Work 
participants vere still active in the program at 12 months after random assignment, as opposed 
to 16 percent for JOBSTART. 

In summary, while only approximate comparisons can be made, it appears that 
JOBSTART achieved its goal of providing young school dropouts with more intensive 
education and training than is usual within the JTPA system. The data also suggest that 
JOBSTART offered an intensity of activity close to that of the Job Corps aid Supported 
Work, which operated through special agencies and had I he sole mission of providing services 
to very disadvantaged individuals. However, for the 40 percent of the JOBSTART sample who 
did not participate or had very few hours of activity, the treatment was unlikely to be intense 
and lengthy enough to move them above the threshold of skills needed to secure significantly 
better jobs than they could before the program. 



median for the Job Corps is not available. 
^*The JOBSTART measure of length of participation included some periods of inactivity in the midst 
of participation, while the Supported Work measure factored these out. As discussed above, however, this 
problem does not appear to have been serious in the JOBSTART data. 
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IIL Moving Behind the Aggregate Participation Measures 

Aggregate measures, however, tell only part of the story. Table 3.5 makes clear that 
JOBSTART was not the same experience for all youths: 40 percent participated for 200 or 
fewer total hours, while nearly 20 percent exceeded 700 hours, the required offering under 
the demonstration (200 hours of education and 500 hours of training). Clear differences in 
average participation also existed among the sites, as discussed later in this chapter. Under- 
standing the sources of these variations in participation is the first step in developing ways to 
improve the design and implementation of the program. 

The following analysis begins with subgroups of JOBSTART participants. It shows that 
while there were differences among subgroups, they did not seem to account for all the 
variation in participation. This implies that factors such as unmeasured differences among 
youths, local employment opportunities, and program characteristics associated with particular 
sites may also have affected participation. 

The key finding on program characteristics is that youths in sites operating sequential/ 
brokered programs tended to have lower rates of participation in occupational skills training, 
although they tended to receive more intensive instruction in basic skills. 

A. Differences in Participation Among Subgroups 

Although JOBSTART participants all satisfied the program's eligibility requirements, 
they varied in gender, age, marital and parental status, criminal records, and educational 
attainment, among other characteristics. Research and operational experience suggest that 
these types of factors can influence participation in programs.^ 

Among JOBSTART participants, two groups are of special concern: males (who have 
often been hard to recruit and retain in programs) and young mothers (a group at risk of long- 
term welfare receipt). As Table 3.7 shows, average total hours and other measures of 
participation were similar for all males and females, although a higher percentage of females 
were active in the twelfth month after random assignment. There were some differences, 
however, for females living with their children, compared to other women: Mothers averaged 



^See, for example^ Public/Private Ventures, 1988, and Mathematica Policy Research, 1985. 
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TABLE 3.7 



PARTICIPATION RATES, HOURS OF PARTICIPATION, AND LENGTH OF STAY, 
BY GENDER AND PARENTAL STATUS 



Women 







Living 


Not Living 




Men 






wi t h Own 


wi th Own 


All 


and 


Activity Measure 


Men 


Lm 1 Q \ ren ) 


Phi 1 fi(r0n\ ^ 


woiTicn 


If Ulnlsri 


Percent participating in 












Any activity 


89.0 


89.6 


Of A 

87.4 


OO.O 


OO. / 


Education 


85.8 


87.6 


84.3 


85.9 


85.9 


Training 


67.4 


69.6 


62.5 


65.9 


66.6 


Education and training 




DO t 


fin 9 


fid 9* 


fid d 


Other activities 


36. 1 


41 .6 


AC o 

45.2 


A'3 A 

4o . 4 


A n n<A"A> 


Average hours in 












Education 


120.2 


119.6 


149.4 


4 1 il 

134.8** 


n oo 1 

128. 1 


Training 


258.5 


250.5 


231.4 


240.7 


248. 9 


Education and training 


378.7 


370.1 


380.8 


IIP £ 
3/5.6 


377 .0 


Other activities 


31.6 


36.2 


47.8 


42.1** 


37.3*** 


All activities 


410.3 


408.1 


428.5 


418.5 


414.8 


Percentage distribution of hours 












in education and training 












None 


11.4 


11.2 


4 1 A 

13.4 


12.3 


11 0 

11.9 


Up to 200 


31.7 


36.8 


32.2 


'iA A 

34. 4 


•J Q o 

00 .2 


201 to 500 


24.0 


19.2 


o 1 

23 .0 


^1 . 1 


99 /I 
22 . 4 


501 to 700 


16.7 


16.0 


13 . 0 


^ A c 


1 c C 
lb . b 


701 or more 


16.2 


16.8 


18.4 


17.6 


17.0 


Total 


100.0 


100.0 


100.0 


99.9 


100.0 


Percentage distribution of hours 












in all activities 












None 


11.0 


lU .4 


ic • 0 


11 ^ 


11.3 


Up to 200 


27.6 


30.8 


28.0 


on A 
29.4 


OQ fi 

2o , D 


201 to 500 


26.7 


24.8 


24. 1 


OA C 

24 . b 


o c c 
2b . b 


501 to 700 


16.2 


15.6 


13 .0 


^ A 1 
14. J 


1 c o 
lb . 2 


701 or more 


18.5 


18 .4 


22 .2 


2U . 4 


1 Q c 
19 . b 


Total 


100.0 


100.0 


99 . 9 


1 nn 1 
lUU . I 


1 nn 1 


A\/orMn0 Ipnfith nf ^t;5\/ imonthsJ 


6.4 


7.2 


).i 


7.2 


6.8** 


Percent still 












participating in month 










78.0*** 


3 


77.6 


79.2 


77.4 


78.3 


6 


52.5 


55.6 


53.6 


54.6 


53.6 


9 


27.4 


33.2 


33.3 


33.3 


30.6 


12 


11.4 


20.4 


21.1 


20.7 


16.4 


15 


7.5 


11.6 


11.1 


11.4 


9.6 


18 


3.4 


6.8 


5.4 


6.1 


4.8 


19 or later 


2.7 


4.4 


4.6 


4,5 


3.7 


Number of experimental s 


438 


250 


261 


511 


949 



(continued) 
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TABLE 3-7 (continued) 



SOURCE: MDRC calculations from JOBSTART enrollment form, MIS, and survey data. 

NOTES: Calculations for this table used data for all experimentals for whom there were 24 
months of follow-up survey data, including those with values of zero for outcomes and those who 
were assigned to JOBSTART but did not participate. 

Distributions may not total 100,0 percent because of rounding, 
A Pearson chi-square statistic was used to test the hypothesis of equal 
distributions. Among all women, the distributions compared are those for women who were living 
with their own child(ren) and those for women not living with their own child(ren), including 
those who were childless, at the time of random assignment. An F-statistic was used to test the 
hypothesis of equal column means. Statistical significance levels are indicated as *** * 1 
percent; ** = 5 percent; * = 10 percent, 

^Includes women who did not have children. 
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somewhat fewer hours of participation, and a slightly higher percentage received fewer than 
200 hours of services. 

Table 3.8 presents average total hours of participation in JOBSTART for several other 
subgroups. Although past research and experience suggest that the characteristics listed in the 
table might affect participation, many of the comparisons do not show significant differences 
in average hours for the groups under review. Youths who had no> ^ .en arrested since age 
16 participated for significantly more hours.^^ But other groupings did not show differences 
in hours. 

B. D lfTercnces in Participation A mon g Sites 

Hours of participation in the sites in the demonstration varied considerably, as shown 
in Table 3.9. Average total hours ranged from a high of 631 for experimental at the Los 
Angeles Job Corps to a low of 166 at CREC in Hartford, a spread of 465 hours. Another 
important aspect of variation was the percentage of experimentals still participating in 
JOBSTART at 12 months after random assignment. This proportion varied from a low of 
zero percent at Chicago Commons and SER/Corpus Christi to a high of 42 percent at 
Allentown. The proportion still active at 18 months ranged from zero percent to 17 percent 
at the Los Angeles Job Corps. Thus, the 24 months of follow-up do not represent the same 
length of post-program follow-up at all sites. 

As noted earlier, this variation could have had several possible sources, such as 
characteristics of the youths, local employment opportunities, and program characteristics.^^ 
With only 13 sites in the demonstration, it is very difficult to isolate the effects on participation 
of the many differences among programs. If, for example, the sites with the most support 
services were also Job Corps sites and also operated a youths-only program, it would be 
impossible to separate out the effects of these individual factors on participation hours. 



-^^The mix of activities did differ by initial reading score. Those testing in the low group averaged 
slightly more hours in education, and had somewhat fewer hours in training, than those in the highest 
group. 

^Differences in the way random assignment interacted with site recruitment efforts, and also in 
attendance reporting, led to variations in participation rates as well. At CET/San Jose, for example, 
services were not available for the first part of the sample for up to one month. Furthermore, as discussed 
in Chapter 2, attendance at a multi-day assessment of occupational training interests was not included in 
reported hours. As a result, 36 percent of the experimentals at that site had no reported hours in program 
services. At other sites, the gap between random assignment and reported program start-up was shorter 
and participation rates were higher. 
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TABLE 3.8 



AVERAGE TOTAL PARTICIPATION HOURS. BY SELECTED CHARACTERISTICS 
OF EXPERIMENTALS AT THE TIME OF RANDOM ASSIGNMENT 



Characteristic 

and Subgroups Average Total Hours Number of Experimental s 

Age 

16-19 405.1 703 

20 or 21 442.3 246 



72 
439 



Ethnicity^ 

White, non-Hispanic 401.7 

Black, non-Hispanic 403.7 

Hispanic 393.5 411 

School grade at time of dropout 

Grade 10 or below 423.6 £48 

Grade 11 or 12 402.6 401 

Reading grade level 

1-6 364.1 258 

7-8 370.7 195 

9 or above 302.3 18 

Gender 

Women 418.5 511 

Men 410.3 438 

Marital status 

Ever married 439.8 88 

Never married 413.2 857 

Parental status 

Women living with own child{ren) 408.1 250 
Women not living with own child{ren)'^ 428.5 261 

AFDC benefits received 

None 409.9 438 

Own AFDC case 448.8 197 

Household AFDC case 431.7 186 

Received occupational 

training within past year 

No 420.0 797 

Yes 387.3 152 

Criminal record 

No arrest since age 16 435.7 805 

Arrested since age 16 297.8*** 144 

Sample size 949 



(continued) 



TABLE 3.8 (continued) 



SOURCE: MDRC calculations from JOBSTART enrollment form, MIS. and survey data. 

NOTES: Calculations for this table used data for all 949 experimental for 
whom there were 24 months of follow-up survey data. Sample sizes reported may fall 
short of this number because of items missing from some sample members' 
questionnai res. 

An F-statistic was used to test the hypothesis of equal means. 
Statistical significance levels are indicated as *** - 1 percent; = 5 percent; * - 
10 percent. 

*The sample also included 27 experimental who were members of other 

ethnic groups. 

'^Includes women who did not have children. 




-86- 



) 



TABLE 3.9 



PARTICIPATION RATES, HOURS OF PARTICIPATION, AND LENGTH OF STAY 
FOR EXPERIHENTALS, BY SITE 



Concurrent 


Sequential/ 
In-House 


Sequential/ 
Brokered 


Total 


Acti v1 ty Measure 


At 1 anta 
Job Corps 


CET/ 

San 

Jose 


East LA 

Ch i cago Connell ey Skills EGOS 
Coffuwns (Pittsburgh) Center (Denver 


SER/ 

Phoenix Corpus 
) Job Corps Christi 


El 

C ntro LA 
(Dallas) Job Corps 


Al lentowr 

(DUTTa 1 OJ 


DC A 
DbA 

(NYC) 


CREC 

(Hartford) 


P^m^nf HA p f 1 r 1 n A f { nn {n 

rCiCCfIL yOl ^ I \» } yd ^ 1 it\f III 






























Education 


81.8 


44.0 


81.1 


96.8 


82.4 


93.5 


83.6 


98.3 


100. U 


77.4 


100.0 


73.8 


88.9 


85.9*** 


Training 


78.8 


60.0 


91 .9 


98.9 


82.4 


77.4 


83.6 


98.3 


48.1 


53.0 


33. 8 


23.1 


15.6 


66.6*** 


Education and traininQ 


/o.o 


40.0 


81 . 1 


96.8 


82.4 


77.4 


83.6 


98.3 


48.1 


53.0 


33.8 


23.1 


15.6 


64 4*** 


Other activttles 


84 .8 


0.0 


0.0 


0.0 


0.0 


0.0 


79.1 


0.0 


100.0 


79.1 


95.9 


75.4 


15.6 


40.0*** 


AwAPAn* nAiiPC in 
nVCi CI^C llUUid III 






























LQuca L 1 on 




7n n 
.o 


72.1 


105.1 


73.4 


126.0 


163.3 


123.9 


141.8 


158.6 


244.3 


144.9 


118.5 




Tpa ( n ( nn 

1 1 Q 1 11 1 TIU 




J JD . 7 


372 .8 


473 .4 


272.0 


147.8 


218.1 


294.2 


179.0 


362.4 


113.1 


66.1 


36.3 


248.9*** 


P/4ii /* A ^ 4 /\n Ann ^PAlnlnn 

cuucaLion ctnii trcuDiriy 




JOT . / 


444.9 


578.7 


345.4 


273.8 


381.4 


418.1 


320.8 


521.1 


357 .4 


211.0 


154.8 


177 0*** 


ULIICf fl^LITILICa 




n n 


0.0 


0.0 


0.0 


0.0 


59.7* 


0.0 


80.2 


105.6 


82.2 


69.0 


10.7 


37.3*** 


All actfvUies 


295.7 


365.7 


444.9 


578.7 


345.4 


273.8 


441.2 


418. 1 


401.0 


630.5 


439.6 


280.1 


165.5 


414.8*** 


Percentttge distribution of 






























nours in eaucaLion 






























AnH t PA i n 1 no 

CIllU vldlllllivj 






























nuf ic 


in 7 

1 O • C 


n 

JU . u 


8.1 


1.1 


17.6 


6.5 


16.4 


1.7 


0.0 


22.6 


0.0 


26.2 


11.1 


1 1 Q 






1 f\ n 


29.7 


19.1 


27.5 


48.4 


31.3 


15.1 


51.9 


27.8 


48.6 


35.4 


64.4 


33.2 


701 to SOO 


7d 7 


n H 

1 J . J 


18.9 


22 .3 


21.6 


24.7 


19.4 


40.3 


19.8 


15.7 


20.3 


23. 1 


17 .8 


22.4 


<J\J I L V / \J\J 


Q 1 


Q 1 


)3.5 


21.3 


15.7 


11.8 


11.9 


42.9 


14.8 


4.3 


14.9 


6.2 


4.4 


15.5 


701 or more 


6.1 


?5.3 


29.7 


36.2 


17.6 


8.6 


20.9 


0.0 


13.6 


29.6 


16.2 


9.2 


2.2 


17.0 


Totsil 


100.0 


99.9 


99.9 


100.0 


100.0 


100.0 


99.9 


100.0 


100.1 


100.0 


100. U 


100.1 


99.9 


100. O**'' 


Percentage distribution 






























of hoijirs in all activities 






























None 


15.2 


3r).o 


8.1 


1.1 


1),6 


6.5 


13.4 


1.7 


0.0 


20.9 


0.0 


24.6 


11.1 


11 1 


Up to 200 


33.3 


Ifi.O 


29.7 


19.1 


27. b 


48.4 


29.9 


15.1 


34.6 


16.5 


37.8 


27.7 


64.4 


28.6 


201 to 500 


30.3 


13.3 


18.9 


22.3 


21.6 


24.7 


20.9 


40.3 


35.8 


21.7 


24.3 


29.2 


15.6 


25.5 


501 to 700 


9.) 


9.3 


13.5 


21.3 


15.7 


11.8 


13.4 


42.9 


8.6 


7.0 


14.9 


3.1 


4.4 


15.2 


701 or more 


12.1 


?5.3 


29.7 


36.2 


17.6 


8.6 


22.4 


0.0 


21.0 


33.9 


23.0 


15.4 


4.4 


19.5 


Total 


100.0 


99.9 


99.9 


100.0 


100.0 


100.0 


100.0 


100.0 


100.0 


100.0 


100.0 


100.0 


99.9 


100.1*** 


Average length of 






























stay (months) 


5 b 


4.4 


4.5 


10.1 


5.4 


7.1 


6.5 






B.l 


10.7 


5.5 


6.3 


6.8*** 



I 

00 

I 
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TABLE 3.9 (continued) 





















Sequential/ 




Sequential/ 












Concurrent 










In-House 




Brolcered 










TFT/ 






East LA 






SER/ 


El 














Atlanta 

nt laiita 


JQII 


Chicago 


Connelley 


Skills 


EGOS 


Phoenix 


Corpus 


Centro 


LA 


Allentown BSA 


CREC 




Activity H^A^ijr# 




.In 


Comnons (Pittsburgh) Cei.>.er 


(Oenver) Job Corps Christi 


(Oallas) Job Corps 


(Buffal 


o) (NYC) 


(Hartford) 


Total 


Pmrrmn^ <f { 1 1 

rcrcfinv 5tl 1 1 






























participating in montn 






























J 


AQ 7 


CT 1 
3/ . J 


64.9 


97.9 


66.7 


79.6 


79.1 


89.1 


85.2 


72.2 


91.9 


63.1 


65.7 


78.0*** 


6 


36.4 


40.0 


45.9 


76.6 


51.0 


57.0 


50.7 


60.5 


44.4 


48.7 


78.4 


38.5 


40.0 


53.6 


9 


21.2 


21.3 


8.1 


64.9 


27.5 


34.4 


32.8 


0.0 


24.7 


38.3 


55.4 


?6.2 


28.9 


30.6 


12 


12.1 


8.0 


0.0 


24.5 


7.8 


21.5 


19.4 


0.0 


6.2 


26.1 


41.9 


15.4 


22.2 


16.4 


15 


6.1 


1.3 


0.0 


17.0 


0.0 


7.5 


6.0 


0.0 


0.0 


23.5 


29.7 


10.8 


11.1 


9.6 


18 


3.0 


0.0 


0.0 


11.7 


0.0 


2.2 


1.5 


0.0 


0.0 


16.5 


10.8 


3.1 


4.4 


4.8 


19 or later 


3.0 


0.0 


0.0 


7.4 


0.0 


0.0 


1.5 


0.0 


0.0 


13.0 


10.8 


1.5 


4.4 


3.7 


Number of experimental s 


33 


75 


37 


94 


51 


93 


67 


119 


81 


115 


74 


65 


45 


949 



SOURCE: MORC calculations from JOBSTART enrollment form. MIS^ and survey data. 



NOTES: Calculations for this table used data for all experlmentals for whom there were 24 ntonths of follow-up survey data. Including those with 
values of zero for outcomes and those who were assigned to JOBSTART but did not participate. 
Oistributions may not total 100.0 percent because of rounding. 

A Pearson chi-square statistic was used to test the hypothesis of equal distributions. An F-statistic was used to test the hypothesis of 
equal column means. Statistical significance levels are indicated as *** « 1 percent; ** > 5 percent; * « 10 percent. 

*The Phoenix Job Corps did not report hours spent by participants In life skills or avocational activities. 



Further, the demonstration was not designed to address this type of question with the same 
rigor provided for comparisons of experimentals and controls. Applicants were randomly 
assigned to the experimental or control group, but there was no random assignment to various 
types of sites, and within each labor market there was usually only one site. This means that 
the power of the random assignment research design applies to differences between 
experimentals and controls (at a site or in the aggregate) and for differences among subgroups 
(as defined by pre-random assignment characteristics). Other types of comparisons, such as 
between types of sites, are inherently less reliable, and the strength of conclusions depends on 
the consistency of results across sites.^ 

Analysis presented in a previous report found that differences in experimentals' 
characteristics among the sites explained only a small part of the differences in average total 
hours. The analysis also found that labor market conditions affected participation: Sites with 
better employment opportunities, other things being equal, had lower average participation 
hours.^ 

The strongest influence on participation in JOBSTART appeared to be program 
structure: whether a site was concurrent, sequentialln-house, or sequential/brokered. 
Participation rates by component, participation hours, and percentage of time in education or 
training all differed among the three types of sites, as detailed in Table 3.9 and summarized 
in Table 3.10.^ Four conclusions about program structure can be drawn: 

• Average hours of participation varied by type of site. Experimentals in 
sequential/in-house sites had the highest average participation hours, 
while those in sequential/brokered sites had by far the lowest because of 
very low average hours in training. 

• The mix of education^ trainings and other activities varied by type of 
site. The concurrent sites, other than two Job Corps sites, did not offer 
the optional "other activities*" and emphasized occupational training; as 



"^See Chapter 6 for a more detailed discussion of the difficulty of making aoss-site comparisons. 
^See Auspos et al., 1989. 

^This could have been because those participating in JOBSTART found a job more easily and left 
the program after fewer hours. AJtematively, sites in labor markets with low unemployment may recruit 
youths who have more unmeasured barriers to employment, are harder to work with in a program, and 
end up with fewer hours of participation. 

^As discussed earlier, eight sites provided concurrent basic education and occupational skills training 
("concurrent" sites); two provided education followed by training f sequential/in-house" sites); and three 
provided education and then referred participants to other agencies for training ("sequential/brokered " 
sites). 
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TABLE 3.10 



PARTICIPATION RATES, HOURS OF PARTICIPATION, AND LENGTH OF STAY, 

BY PROGRAM STRUCTURE 



Activity Measure 



Percent participating in 
Education 
Training 

Education and training 
Other activities 



Concurrent 



84.9 
85.2 
81.5 
14.2 



Sequential/ 
In-House 



86.7 
51.0 
51.0 
87.8 



Sequential / 
Brokered 



88.0 
25.5 
25.5 
69.0 



Total 



85.9 
66.6*** 
64.4*** 
40.0*** 



Average hours in 
Education 
Training 

Education and training 
Other activities 
All activities 



103.7 
291.3 
395.0 
10.0 
405.0 



151.7 
286.6 
438.3 
95.1 
535.6 



178.4 
77.7 

256.1 
60.0 

316.2 



128.1*** 
248.9*** 
377.0*** 
37.3*** 
414.8*** 



Percentage distribution of hours 
in education and training 
None 

Up to 200 
201 to 500 
501 to 700 
701 or more 
Total 



11.4 
26.9 
24.8 
19.9 
17.0 
100.0 



13.3 
37.8 
17.3 
8.7 
23.0 
100.1 



12.0 
47.8 
20.7 
9.2 
10.3 
100.0 



11.9 
33.2 
22.4 
15.5 
17.0 
100.0*** 



Percentage distribution of hours 
in all activities 
None 

Up to 200 
201 to 500 
501 to 700 
701 or more 
Total 



20.9 
16.5 
21.7 
7.0 
33.9 
100.0 



13 
29 
20 
13 
22 
100 



1.7 

15.1 
40.3 
42.9 
0.0 
100.0 



11.3 
28.6 
25.5 
15.2 
19.5 
100.1*** 



Average length of 
stay (months) 

Percent still participating in month 
3 
6 
9 

12 
15 
18 

19 or later 



6.4 



78.9 
55 5 
27.2 
12.3 
5.3 
2.6 
1.6 



7.1 



77.6 
46.9 
32.7 
17.9 
13.8 
9.7 
7.7 



7.8 



75.5 
54.9 
38.6 
27.7 
18.5 
6.5 
6.0 



6.8" 



78.0*** 
53.6 
30.6 
16.4 

9.6 

4.8 

3.7 



Number of experimental 569 196 184 949 
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TABLE 3,10 (continued) 



SOURCE: MDRC calculations from JOBSTART enrollment form, MIS, and survey data, 

NOTES: Calculations for this table used data for all experimental for whom there were 24 
months of follow-up survey data, including those with values of zero for outcomes and those who 
were assigned to JOBSTART but did not participate. 

Distributions may not total 100.0 percent because of rounding. 

A Pearson chi -square statistic was used to test the hypothesis of equal 
distributions. An F-statistic was used to test the hypothesis of equal column .neans. 
Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 



a result, average training hours for experimentals amounted to 72 percent 
of average total hours.^^ The sequential/brokered sites emphasized 
education and other non-training services, all of which were provided 
in-house. They had the highest average hours in education; training 
hours were only about 25 percent of average total hours. The 
sequential/in-house sites had the highest total average hours, hours in 
education and other non-training activities approximating those of the 
sequential/brokered sites, and hours in training like those of the 
concurrent sites. 

• Sequential/brokered sites had difficulty moving participants from 
education to training* Only 26 percent of participants at sequen- 
tial^rokered sites made the transition to occupational training, although 
those who made the transition did receive substantial training* This low 
rate of participation in training occurred because of the difficulty of 
linking participants with other organizations, in part because of the 
nature of typical JTPA Title IIA contracts. Possibly, it also arose 
because participants in these sites (which were primarily basic education 
organizations) were more interested in receiving a GED than 
occupational training. 

• These relationships do not appear to have been the result of measured 
differences in participant characteristics or local employment 
opporiunities. Even after adjustments for measured differences in 
participant t;haracteristics and local employment opportunities, these 
patterns of participation among sites with different program structures 



While these three categories of sites do clarify patterns of participation, the sites within 
each category were clearly not identical. Among the concurrent sites, EGOS in Denver stood 
out with especially low hours - possibly because of its very large size, which could have left 
the JOBSTART youths feeling isolated and disconnected from the program. CREC in 
Hartford, among the sequential^rokered sites, had very low hours because it scheduled only 
three hours of education per day and very few experimentals participated in training. 
Furthermore, CREC offered limited support services and moved several times during the 
demonstration, which disrupted program operations. The high total hours for sequential/ 
in-house sites were primarily owing to the Los Angeles Job Corps, which had the highest 



^'^The percentage of average total hours is calculated by dividing average training hours by average 
total hours. Percentages that follow are similarly calculated. 

^Whcn dummy variables for type of site were added as independent variables to a regression equation 
with individual demographic characteristics and a measure of local employment opportunities, the 
relationships still held. 
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average hours among all sites. El Centro in Dallas, the other site in this category, ranked 
slightly below the average foi all sites in total hours. 

IV. Summary of the Features of the Sites 

The diversity of the sites wthin the tjeneral framework of the JOBSTART guidelines 
has been a theme of this chapter But before moving on to consider the impacts of these 
programs, it is useful to summarize in one place the key features of each local JOBSTART 
program. Table 3.11 groups the sites by program structure (concurrent, scquential/in-house, 
sequential/brokered) and uses three types of measures to summarize JOBSTART 
implementation. First, it describes participation in JOBSTART for experimental by presenting 
average UHal hours; average length of stay in the program; average hours per month in the 
program; and average hours of education and training. (Note that each of these measures 
includes the 11 percent of experimental who did not participate in the program.) All of these 
items have been included in previous tables except for average hours per month, which is a 
measure of the extent to which program services were concentrated or spread out over time. 

Table 3,11 also includes more subjective ratings of implementation, including a rating 
of the level of initial screening done by each site at intake. Initial screening was greatest at 
the three Job Corps Centers (which had special entrance criteria) and Chicago Commons 
(which had special requirements for entering its vocational training program). The table also 
includes ratings of job placement and support services (drawn from Table 3.4) and an overall 
assessment of JOBSTART implementation.^ 

Finally, the cost per experimental in each site's research sample, listed in the right 
column of this table, adds to the description of program implementation already presented in 
this chapter."^^ Most programs tend to fall in the range of $4,500 to $6,500, but several fall 



"^^Chapter 6, which discusses differences in impacts among the sites, contains a further summary table 
of each site*s program. 

•^Sec Appendix E for the details of these cost data, which include items not normally part of program 
budgets, such as the value of donated goods and services. These cost figures are not the ones that would 
be produced as part of a benefit-cost analysis that compares the impact of the program to the incremental 
cost of ihe services received by experimentals above the level of services received by controls. As Ls 
discussed in Chapter 4, the controls received substantial non-JOBSTART services. Thus the impact 
figures estimate the effect of the services received by experimentals above this base of services. Costs 
used to calculate a benefit-cost ratio would measure the resource cost of the extra services received by 
experimentals. The cost figures reported here are gross program costs, before subtracting the cost of 
services received by controls. 
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TABLE 3.11 

SUMMARY OF JOBSTART IMPLEMENTATION, BY SITE 



JOBSTART 

Length Average Hours of Level of Rating of Overall Operating 

Total of Stay Hours Initial Job Support Rating of Costs per 

Site Hours (Months) per Month Education Training Screening Placement Services Implementation Experimental ($)* 



Concurrent 


405 


6.4 


GO 


104 


291 





— 


— 


... 


... 


Atlanta Job Corps 


296 


5.5 


50 


95 


liO 


High 


Low 


High 


Low 


4,100'' 


CET/San Jose^ 


366 


4.4 


74 


2gd 


337 


low 


High 


Medium 


High 


2,000 


Chicago Comons^ 


445 


4.5 


83 


72^ 


373 


High 


High 


Medium 


High 


6,400 


Conncllcy (Pittsburgh) 


579 


10.1 


54 


105 


473 


Medium 


Medium 


High 


High 


5,200 


East LA Skills Center 


345 


5.4 


55 


73 


272 


Medium 


Low 


Medium 


Medium 


4,900 


EGOS (Denver) 


274 


7.1 


33 


126 


14B 


Low 


Low 


Medium 


Low 


2,000*^ 


Phoenix Job Corps 


441 


6.5 


60 


163 


218 


High 


High 


High 


High 


4,700'' 


SER/Corpus Christi 


418 


5.2 


76 


124 


294 


Medium 


Low 


High 


Medium 


2.100 


Sequent ial / in-house 


536 


7.1 


66 


152 


287 












El Centro (Dallas) 


401 


5.8 


60 


142 


179 


Medium 


Medium 


Medium 


High 


5,100 


LA Job Corps 


631 


8.1 


71 


159 


362 


High 


Medium 


High 


High 


5,700*^ 


Sequential /brokered 


316 


7.8 


37 


178 


78 












Allentown (Buffalo) 


440 


10.7 


39 


244 


113 


Medium 


Low 


High 


Medium 


5,900 


BSA (NYC) 


280 


5.5 


45 


145 


66 


Low 


low 


Medium 


Low 


7,500 


CREC (Hartford) 


166 


6.31 


23 


119 


36 


Low 


Low 


Medium 


Low 


5,200 


All sites 


415 


6.8 


57 


128 


249 
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SOURCES: MDRC calculations from JOBSTART enrollment form, MIS, and survey data (participation figures); MDRC operations staff (implementation ratings); 
Appendix E (costs). 

NOTES: *Thesc cost estimates are preliminary and therefore rounded to the nearest ':D0. All costs are in 1986 dollars. 

''The estimates do not include the cost of providing on-site medical and dental services. The value of these services per JOBSTART experimental 
was approximately $400 in the Atlanta site, $400 in Phoenix, $600 in Los Angeles, and $24 in Denver. 

^In this site, education and training were more integrated than in other sites, and staff strongly emphasized training over passing the GEO 

examination. 

^!n this site, some education hours are included in the training component hours. 
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well outside this range. The variation in program costs is substantial, reflecting several 
factors: 

• Enrollment levels. Programs serving a higher volume of participants 
relative to their institutional capacity can spread the fixed costs of 
operation over many people, thereby lessening the average cost. EGOS 
in Denver, a large adult vocational school with more than 15,000 
students, thus had very low costs per experimental. In contrast, BSA in 
New York City had difficulty enrolling enough students to fill all of its 
available slots and as a result had very high costs per experimental. 

• Intensity of the planned components. More ambitious training or 
support services cost more per month of operation. This was the case 
at Chicago Commons, where the training options were often quite 
intensive. 

• Average length of participation. Since costs per experimental depend 
on the cost per unit of service ind the number of units used, sites with 
longer participation, such as ti,e Los Angeles Job Corps, have higher 
costs. 

• Staff salary levels. Personnel costs are the most important part of costs, 
so an agency's salary structure and level are major determinants of cost. 
The low average cost of SER/Corpus Christi is partly explained by that 
site's relatively low personnel expenses. 

The following two chapters present program impacts, for the sample as a whole and for 
key subgroups defined by individual characteristics. The final chapter of the report returns to 
the issue of site variation, in reviewing the pattern of program impacts among the 13 sites. 
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CHAPTER 4 

RECEIPT OF EDUCATION AND SKILLS TRAINING 



AND TOBSTARrS IMPACTS ON EDUCATIONAL ATTAINMENT 



The goals of the JOESTART program included inc 5»i5ing participation in education and 
training activities by a group who otherwise would be little served, and thereby enhancing their 
educational attainment, employability, and long-term earnings. Chapter 3 described the 
experimental group's participation in JOBS'^^ART activities.^ This chapter takes the story a 
step farther by comparing the participation of these experimentals in JOBSTART (and other 
programs) with the activities of those in the control group, to determine whether the offer of 
JOBSTART services actually led to an increase in participation over what would have occurred 
otherwise.^ As will be shown, the youths in the experimental group did participate in more 
education and training than did those in the control group, but the control group was not 
unserved. Chapters 5 and 6 examine the extent to which this increase in services has so far 
led to improvements in employment, earnings, welfare receipt, and other longer-term 
outcomes.^ 

This and succeeding chapters rest on the random assignment research design, described 
briefly in earlier chapters, to estimate the difference the JOBSTART program made in the lives 
of those young people given access to it. To evaluate these differences (often called program 
"impacts"), it is necessary to answer two basic questions. First, on average, what happened to 
those who were offered the program - in this context, the "experimental" group? Second, on 
average, what would have happened to them had they not been offered the program, here 
represented by the experience of the "control" group?'* The average effect, or "impact," of a 
program is the difference between the two groups in the m"iy outcomes of interest. 



^This was based on attendance information reported by the JOBSTART program operators for those 
in the experimental group. 

^In addition to using the JOBSTART attendance data from the program MIS, this chapter relies on 
survey data for both experimentals and controls gathered 12 and 24 months after random assignment. The 
survey data include participation in other programs as well as in JOBSTAPT. 

^Chapter 5 focuses on the story for the full impact sample and for subgroups defined by individual 
characteristics such as gender, while Chapter 6 treats cross-site differences in impacts. 

''Since, as shown in Chapter 2, assignment to JOBSTART was random, there were no systematic 
differences between experimentals and controls at enrollment, and outcomes for controls could be used 
to measure what would have happened to experimentals without the program. 
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This chapter addresses several key evaluation questions: 

* Did the planned diRerential between experimentals and controls in the 
receipt of education and occupational skills training materialize? Was 
this differential maintained during the second year of follow-up, or had 
controls begun to catch up to experimentals? 

• How did service receipt and the differential in service receipt compare 
for important subgroups such as men, young mothers, and other women? 

* Did JOBSTART produce gains in educational attainment, as measured 
by receipt of high school diplomas and passing of the GED examination, 
during the first two years after random assignment? Were controls 
catching up to the experimentals in educational attainment during the 
second year? 

• How did the educational attainment differential compare for important 
subgroups such as men and women? 

To summarize the basic findings in this chapter, access to JOBSTART did substantially increase 
the experimentals' participation in education and training activities, raising their rates and 
average hours of participation well above those of controls, almost half of whom were also 
active in these types of programs. This greater service receipt among experimentals occurred 
for all important subgroups in the sample; it led to substantially higher rates of receipt of a 
GED or high school diploma among experimentals than among controls for the full impact 
sample and for most subgroups. 

1. Receipt of Educatiop and Training bv Experimentals and Gintrols 

The JOBSTART control group was used as a benchmark for measuring program 
impacts, but if most controls received services similar to those received by experimentals, the 
benchmark would have been useless, and it would be very difficult to evaluate JOBSTART/** 
Because JOBSTART targeted disadvantaged dropouts with poor reading skills (a group thought 



^The service receipt differences reported here are calculated by comparing the experiences of 
experimentals and controls as is done for analysis of the impact of JOBSTART on outcomes such as 
educational attainment and employment These service receipt differences are not normally thought of 
as program impacts because they are an intermediate step in reaching the final program goals. This 
section uses the terminology of program impacts at times when it simplifies the presentation of the results. 
This terminolo^ is useful here because the chapter discusses differences ?:,mong subgroups in the size of 
these experimental-control differences; the use of the term impact for the experimental-control difference 
simplifies sentences. 
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hard to serve within the JTPA system), demonstration planners anticipated that JOBSTART 
controk would not be served to any great extent. 

However, JOBSTART controls were expected to receive some services. Even though 
many performance-driven programs seem to screen out people with low reading levels, 
JOBSTART recruits (including those who ended up in the control group) were more 
determined to receive help than was the average school dropout. As a result, at some point 
within the first 24 months following random assignment, 44 percent of controls found remedial 
or occupational instruction elsewhere. The program impacts on educational attainment, 
employment, and earnings presented here, therefore, are the incremental impacts of 
JOBSTART over the mix of services available throughout the community to a group of poor 
readers, on their own initiative, without special referral from JOBSTART site operators. 

A. In-Program and Post-Propram Outcomes 

All the events tracked by the JOBSTART program attendance reporting system and the 
follow-up survey (including program participation, GED receipt, employment, and other 
important outcomes) were reckoned from the date of random assignment, not the date of 
termination from the program.^ There was a great deal of variation in lengths of stay in 
JOBSTART. However, an approximate dividing line between predominantly in-program and 
predominantly post-program periods is no earlier than about the end of the twelfth month 
after random assignment. About 84 percent of experimentals had stopped participating in 
JOBSTART by then. (See Table 3.5.) 

Table 4.1 shows that, over the two-year period as a whole, 92.7 percent of experimentals 
a ad 44.2 percent of controls received some education or training, for an impact of 48.4 
percentage points. This impact was statistically significant, that is, too big to have arisen 
entirely by chance. As shown in Figure 4.1, the proportion of experimentals in programs, 
mainly JOBSTART, was highest during the first three months and fell rather steadily over 
time, to 11.2 percent during month 24. The proportion of controls in programs was much 
smaller at the beginning of the follow-up period, peaked at about 15 percent during months 



^is is a different approach from that used in the Job Corps study, in which follow-up began at 
termination from the program. As will become clear In Chapter 5, starting follow-up with entry into the 
program allows a careful analysis of the forgone earnings caused by participation in the program. But 
this difference in approach complicates the comparison of JOBSTART and Job Corps impact findings. 
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TABLE 4.1 



IMPACTS C I RECEIPT OF EDUCATION OR TRAINING 
THROUGH MONTH 24 



Outcome and Fol low-Up Period 



Experimentals Controls Differonce 



Ever received any education 
or training, months 1-24 



92.7% 



44.2% 



48.4^ 



0.000 



Ever received any education 
or training, months 1-12 



90.7 



29.0 



61.6* 



0.000 



Ever received any education 
or training, months 13-24 



34.3 



30.7 



3.6* 



0,089 



Sample size 



949 



890 
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FIGURE 4.1 

MONTHLY RECEIPT OF EDUCATION 
OR TRAINING, BY RESEARCH STATUS 
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(continued) 



TABLE AND FIGURE 4.1 (continued) 



SOURCES FOR TABLE AND FIGURE 4.1: MDRC calculations from JOBSTART enrollment 
form, MIS, and survey data (Table 4.1); Appendix Table F.l (Figure 4.1). 

NOTES FOR TABLE AND FIGURE 4.1: Calculations for this table and figure used 
data for all sample members for whom there were 24 months of follow-up survey data, 
including those with values of zero for outcomes and those who were assigned to JOBSTART but 
did not participate. 

"Any education or training" includes JOBSTART and non-JOBSTART education, 
occupational skills training, and related activities. 

Random assignment did not always take place on the first of the month. For some 
sample members, the month of random assignment may be less than a month, beginning with the 
date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means 
from linear analysis of covariance procedures controlling for up to 31 kinds of difference 
in characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and 
Appendix Table B.4). There may be slight discrepancies in reported sums and differences of 
these adjusted means because of rounding. 

^The column labeled "p" is the statistical significance level of the difference 
between experimental and control group outcomes: that is, p is the probability that average 
outcomes are different only because of random error. A two-tailed t-test was applied to 
each difference between average experimental and control group outcomes. Statistical 
significance levels are indicated as *** « 1 percent; ** « 5 percent; * « 10 percent. 
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10 through 13, and then soon reached a plateau at about 10 to 12 percent, about even with 
the level to which experimentals had fallen. 

Breaking the follow-up period into a predominantly in-program period (months one 
through 12) and a predominantly post-program period (months 13 through 24) reveals a more 
dramatic pattern of experimental and control differences. Table 4.1 shows that during the first 
12 months of follow-up, 90.7 percent of experimentals participated in education or training 
compared to 29 percent of controls, for an in.pact of 61.6 percentage points. During the 
second 12 months, 34.3 percent of experimentals and 30.7 percent of controls were in 
education or training, for an impact of 3.6 percentage points. Thus, there is no evidence that 
controls are catching up with experimentals in education and training. On the contrary, 
experimentals had not fallen below controls in education and training even in the second year 
of follow-up; both groups had stabilized at about the same rate of service receipt by the end 
of the second year. 

Hours of education or training followed a very similar pattern. Table 4.2 shows that 
over the two-year period, experimentals received an average of 619 hours, while controls 
received an average of 250 hours, for an impact of 369 hours. It is important to remember 
that these average figures for the two groups include those who did not participate and, 
therefore, had zero hours. As Figure 4.2 shows, average experimental hours peaked at about 
70 hours per month during month two and then fell steadily, while control hours, always much 
lower, peaked during month nine and then stabilized at about 10 hours per month. 

Among those 92.7 percent of experimentals and 44.2 percent of controls who received 
any services during the two-year period - that is, excluding those who received no services - 
experimental averaged 668 hours, and controls 'ivcraged 566 hours.^ Because so many fewer 
controls received services, and those who did find services on their own were probably quite 
motivated, this difference in duration for those who received services may undei^state the true 
advantage in intensity for served experimentals. The served experimentals who were directly 
comparable in motivation to the served controls might well have been those who received 
many more than the average hours for all served experimentals. 

Thus, the planned service differential between experimentals and controls materialized. 
Not only did experimentals receive education and training at vastly higher rates than did 



^This is not shown in the table or figures. 
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TABLE 4.2 



IMPACTS ON TOTAL HOURS OF EDUCATION OR TRAINING RECEIVED 

THROUGH MONTH 24 



Outcome and Fol low-Up Period 

Total hours of education or 
training received, months 1-24 



Experimental Controls Difference 



619.16 



250.14 



369.02*** 



0.000 



Total hours of education or 
training received, months 1-12 



439.02 



130.06 



308.96*** 



0.000 



Total hours of education or 
training received, months 13-24 



180.14 



120.08 



60.06 



0.001 



Sample size 



949 



890 



FIGURE 4.2 

MONTHLY AVERAGE" HOURS OF EDUCATION 
OR TRAINING RECEIVE-.D, BY RESEARCH STATUS 
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(continued) 



TABLE AND FIGURE 4,2 (continued) 



SOURCES FOR TABLE AND FIGURE 4.2: MDRC calculations from JOBSTART enrollment 
fornix MIS, and survey data (Table 4,2); Appendix Table F*2 (Figure 4,2) • 

NOTES FOR TABLE AND FIGURE 4.2: Calculations for this table and figure used 
data for all sample members for whom there were 24 months of follow-up survey data, 
including those with values of zero for outcomes and those who were assigned to JOBSTART but 
did not participate. 

For experimental, "hours of education or training" Include JOBSTART hours from 
MIS data and non-JOBSTART hours from survey data. 

Random assignment did not always take place on the first of the month. For some 
sample members, the month of random assignment may be less than a month, beginning with the 
date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means 
from linear analysis of covariance procedures controlling for up to 31 kinds of difference 
in characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and 
Appendix Table B.4). There may be slight discrepancies in reported sums and differences of 
these adjusted means because of rounding. 

^The column labeled "p" is the statistical significance level of the difference 
between experimental and control group outcomes: that is, p is the probability that average 
outcomes are different only because of random error. A two-tailed t-test was applied to 
each difference between average experimental and control group outcomes. Statistical 
significance levels are indicated as = 1 percent; = 5 percent; * = 10 percent. 
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controls throughout much of the 24-inonth follow-up period, but they also on average received 
many more hours. However, as noted earlier, the control group did receive some other 
services in the community. Thus, the fmdings presented in Chapters 4 through 6 represent the 
incremental impacts of JOBSTART above the existing level of service. 

B. Differences in Receipt of Education and Training bv Men and Women 

Table 4.3 results from splitting the sample into three groups (men, women living with 
their own children, and other women) and repeating the calculations made for the full sample 
in Tables 4.1 and 4.2. In general, Table 4.3 shows that among the experimental group, men, 
women living with their own children, and other women tended to be active in all types of 
education and training activities in approximately similar amounts in the first year.® However, 
there were clear differences in participation among the control groups for the three subgroups: 
Men in the control group participated in education and training less than did women living 
with their own children, who in turn participated less than other women. The larger 
differences between experimental and controls for men among those who ever participated and 
in average total hours, therefore, result largely from the lower level of activity among men in 
the control group. 

To summarize the service receipt differences shown in Table 4.3: During the two-year 
follow-up period, 92.8 percent of men in the experimental group and 36.6 percent in the 
control group were active in education or training, for a difference of 56.1 percentage points. 
This resulted in a difference of 424 average total houi^ between men in the experimental and 
control groups. For women living with their own children, 45.7 percentage points more 
experimental were active during the two-year follow-up period, and experimentals averaged 
344 more hours than did controls. For other women, the comparable figures were 38.4 
percentage points and 303 hours. 



^is finding is based on analysis of the proportion of experimentals in each group who *'cver 
participated** in an activity during the period in question. The finding holds during the first year of follow- 
up and over the entire 24-month follow-up period. In the second year of follow-up (and over the entire 
two-year follow-up period), more "other women* paiiicipated in an activity (registering the highest average 
total hours in the second year). Women living with their own children had the next highest average total 
hours, and men had the lowest. These differences were largely attributable to differences in participation 
in non-JOBSTART activities. As reported in Chapter 3, participation in JOBSTART activities was similar 
among the three groups. 
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TABLE 4.3 



IMPACTS ON RECEIPT OF EDUCATION OR TRAINING THROUGH MONIH 24, 
BY GENDER AND PARENTAL STATUS 



Subgroup, Outcome, 
and Fol low-Up Period 



Experimental s Controls Difference 



Men 



Ever received any education 
or training (%) 

Months 1-24 

Months 1-12 

Months 13-24 

Total hours of education or 
training received 

Months 1-24 

Months 1-12 

Months 13-24 

Sample size 

Uomen living with own child(ren) 

Ever received any education 
or training (%) 

Months 1-24 

Months 1-12 

Months 13-24 

Total hours of education or 
training received 

Months 1-24 

Months 1-12 

Months 13-24 

Sample size 

\/lomen not living with own 
child(ren), including those 
who did not have any 

Ever received any education 
or training (%) 

Months 1-24 

Months 1-12 

Months 13-24 

Total hours of education or 
training received 

Months 1-24 

Months 1-12 

Months 13-24 

Sample size 



92.8 
90.5 
29.5 



604.57 
444.93 
159.64 



438 



93.6 
9L! 
38.4 



599.32 
411.36 
187.96 



250 



92.5 
90.5 
39.9 



672.35 
456.32 
216.03 



36.6 
24.1 
23.3 



180.12 
94.95 
85.16 

433 



47.9 
'.9.2 
35.5 



255.57 
126.06 
129.52 

234 



54.1 
38.4 
38.5 



369.05 
201.57 
167.48 



56.1*** 
66.4*** 
5 9** 



424.45*** 
349.98*** 
74.47*** 



45,7*** 
61.9*** 
2.9 



343.75*** 
285.30*** 
58.45* 



38.4*** 
52.1*** 
1.4 



303.30*** 
254.75*** 
48.55 



0.000 
0.000 
0.038 



0.000 
0.000 
0.002 



0.000 
0.000 
0.522 



0,000 
0.000 
0.088 



0.000 
0.000 
0.758 



0.000 
0.0 JO 
0.202 



261 



223 



ERIC 



-105- 



(continued) 



TABLE 4.3 (continued) 



SOURCE: MDRC calculations from JOBSTART enrollment form, MIS, and survey data, 

NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate, 

"Any education or training" includes JOBSTART and non-JOBSTART education, 
occupational skills training, and related activities. For experimental s , "hours of 
education or training" include JOBSTART hours from MIS data and non-JOBSTART hours from 
survey data. 

Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted 
means from split-file linear analysis of covariance procedures controlling for up to 29 
kinds of difference in characteristics before random assignment (see Ostle, 1975, p, 461; 
Cave, 1987; and Appendix Table B,4), There may be slight discrepancies in reported sums and 
differences of these adjusted means because of rounding, 

*The column labeled "p" is the statistical significance level of the 
difference between experimental and control group outcomes: that is, p is the probability 
that average outcomes are different only because of random error, A two-tailed t-test was 
applied to each difference between average experimental and control group outcomes. 
Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 
percent. 



C. Differences In Receipt of Education and Training bv People with Selected 
Ciiaracteristics 

The primary goal of the evaluation is to estimate the difference that access to the 
JOBSTART program makes for its target population (in other words, the program's impacts). 
While the size of the JOBSTART sample is large enough for finding policy-relevant overall 
impacts, it provides considerably less statistical power for estimating subgroup impacts and 
differences in impacts among subgroups. Keeping this limitation in mind, this section presents 
an analysis of he difference in service receipt among experimentais and controls in various 
subgroups of youths and compares the size of this difference for the selected subgroups. 

To summarize the findings of this section: The service receipt differences among 
experimentais and controls observed for the full sample were present and large for virtually all 
"important subgroups. Observed differences in service receipt impacts among subgroups 
primarily reflected variation in the level of service receipt of controls. 

Table 4.3 presents impacts calculated using the same methods as were used for the full 
sample: Impacts for the three subgroups were calculated by comparing the experiences of the 
experimentais in each group to those of the controls in the group. Table 4.4 uses a different 
method to calculate within-subgroup impacts and between-subgroups impact differences for the 
most important measure of program activity: receipt of education or training during the two 
years of follow-up available so far. It presents impacts for each subgroup, controlling for 
differences other than the characteristic used to define the subgroup.^ 

For example, the first three rows of Table 4.4 present impacts for women and men using 
statistical techniques to control for gender differences, such as in employment experience, 
educational levels, ethnicity, and parental status. It thus shows a comparison of the impacts 
by gender with other characteristics held constant. The first row of Table 4.4, in the column 
labeled "subgroup impact difference," shows that the impact for women was 13.4 percentage 
points below the impact for men, and this difference in service receipt impacts was statistically 



impact estimates in this table were produced by conducting a two-way analysis of covariance, 
controlling for differences in pre-random assignment characteristics other than the characteristic used to 
define the subgroup. See Ostle, 1975, p. 461. The adjustments were done using a linear regression model. 
Characteristics that affect outcomes and impacts with a nonlinear relationship are not controlled for with 
this procedure. And no such procedure can control for unmeasured characteristics that affect outcomes 
and impacts. The adjusted outcomes for men, women living with their own children, and other women 
presented in Table 4.3 are very similar to the unadjusted outcomes in Table 4.2, but the adjustments do 
make more of a difference for many of the other subgroups listed in the table. 
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TABLE 4.4 



IMPACTS ON RECEIPT OF EDUCATION OR TRAINING 
THROUGH MONTH 24, BY SELECTED CHARACTERISTICS 
AT THE TIME OF RANDOM ASSIGNMENT 







Ever Received Any Education 














or Training, Months 1-24 






















Subgroup 






Sampl e 






Subgroup 




Impact 




Characteristic and Subgroups 


Size 


Experimental s 


Controls 


Impact 




Difference'' 


P» 


Gender 












- 1 J . 4 


O.OOu 


Women 


968 


93.5V 


51.4V 


42 . 1*** 


0.000 






Men 


871 


91.9 


36.4 


55.5*** 


0.000 





— 


Ethnicity 















0.260 


White, non-Hispanic 


155 


93.8 


38.9 


54.9*** 


0.000 





— 


Black, non-Hispanic 


840 


91.9 


47.1 


44 7*** 


0.000 





— 


Hispanic 


783 


92.7 


42.1 


50.6*** 


0.000 





— 


Other 


61 


101.8 


47.0 


54.8*** 


0.000 





— 


Ethnicity, by gender 












*** 


0.003 


Women 
















White, non-Hispanic 


o4 


92.4 


42.8 


An * * 


n nnn 

0 . 000 






Black, non-Hispanic 


445 


94.5 


54.2 


40.3*** 


0.000 


« 


. «• . 


Hispanic 


413 


92.0 


50.9 


41 . 1*** 


0.000 








Other 


26 


106.6 


45.0 


61.5*** 


0.000 








Hen 
















White, non-Hispanic 


71 


96.1 


34.1 


61.9*** 


0.000 





..^ 


Black, non-Hispanic 


395 


88.9 


39.2 


49.6*** 


0.000 





..^ 


Hispanic 


370 


93.7 


32.6 


61.1*** 


0,000 


... 


.... 


Other 


35 


97.0 


46.7 


50.4*** 


0.000 


— - 





Parental status 














0.001 


Women living with own 
















chi Id(ren) 
















No 


484 


93.7 


55.8 


38.0*** 


0.000 






Yes 


484 


90.5 


44.3 


46.1*** 


0.000 






Men who have own child(ren) 
















No 


765 


94.1 


38.3 


55.7*** 


0.000 






Yes 


106 


89.4 


35.9 


53.5*** 


0.000 
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Ever Received Any Education 














or Training. Months 1-24 






















Subgroup 






Sampl e 






Subgroup 




Impact 




Characteristic and Subgroups 


Size 


Experimental s 


Controls 


Impact 




Di f f erence'' 




Employed within past year 












-6.0* 


0.097 


No 


870 


92.4% 


47.2% 


45.3*** 


0.000 


— 


— 


Yes 


969 


92.9 


41.6 


51.3*** 


0.000 


— 


— 


Prior employment, by gender 














0.001 


Women employed within 
















past year 
















No 


547 


96.0 


52.9 


43.1*** 


0.000 


— 


— 


Yes 


421 


90.9 


50.2 


40.8**'* 


0.000 






Men employed within past year 














No 




88.5 


39.6 


49.0*** 


0.000 




^ _ _ 


Yes 


548 


93.4 


34.1 


59.3*** 


0.000 


— 


— 


Left school in grade 11 or 12 












0.9 


0.811 


No 


1,078 


93.6 


44.8 


48.8*** 


0.000 


— 


— 


Yes 


761 


91.4 


43.5 




0.000 








Received occupational training 
















within past year 












-4.9 


0.302 


No 


1,529 


92.7 


45.2 


47.5*** 


0.000 




... 


Yes 


310 


92.2 


39.8 


52.5*** 


0.000 


— 


— 


Age 












4.9 


0.230 


16-19 


1,359 


93.4 


43.7 


49.7*** 


0.000 


— 


— 


20 or 21 


480 


90.4 


45.7 


44.8*** 


0.000 




... 


Age, by gender 














0.002 


Women 
















16-19 


710 


94.0 


50.9 


43.0*** 


0.000 


— 


— 


20 or 21 


258 


92.5 


52.9 


39.6*** 


0.000 






Men 
















16-19 


649 


93.1 


36.1 


57.0*** 


0.000 






20 or 21 


222 


88.2 


37.1 


51.1*** 


0.000 
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TABLE 4.4 (continued) 







Ever Received Any Education 
or Training. Months 1-24 










Lnaracieri 5 ti c and subgroups 


Sample 
Si ze 


Experimentals 


Controls 


Subgroup 
Impact 




Subgroup 
Impact 
Difference^ 




Marital status 
Ever married 
Never married 


174 

1,665 


97.4V 
92.2 


47.4% 
43.9 


50. 0*** 
48.3*** 


0.000 
0.000 


1.7 


0.779 


Living in own household or 
with boy/girlfriend 

No 

Yes 


1,500 
339 


93.3 
89.7 


43.6 
46.9 


49. 7*** 
42.7*** 


0.000 
0.000 


7.0 


0.134 


Own AFDC case or receiving 
General Assistance 
No 

T J 


1,344 
495 


93.3 
90.7 


42.3 
49.4 


51.1*** 
41.3*** 


0.000 
0.000 


9.8** 


0.016 


Own AFDC case 
No 
Yes 


1,446 
393 


92.7 
92.4 


42.0 
52.4 


50.7*** 
39.9*** 


0.000 
0.000 


10.8** 


0.014 


Receiving Food Stamps 
No 
Yes 


1,143 
696 


95.5 
88.1 


42.9 
46.4 


52.5*** 
4 1 . 7*** 


0.000 
0.000 


10.9*** 


0.003 


Arrested since agt 16 
No 
Yes 


1 ,567 
2/2 


92.5 
93.3 


45.1 

39.2 


54.1*** 


0.000 
0.000 


-6.6 


0.194 


Lived with both parents at 
age 14 

No 

Yes 


1,198 

641 


91.8 
94.3 


44.6 
43.6 


47.2*** 
50.7*** 


0.000 
0.000 


-3.5 


0.352 
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Ever Received Any Education 
or Training, Months 1-24 








Characteristic and Subgroups 


Sample 
Size 


Experimental s Control s 


Subgroup 
Impact 




Subgroup 

Impact 
Difference^ p" 


Reason for leaving regular 
high school 

School -rel ated 

Job-related 

Other 


886 
182 
771 


92.9% 41.1% 
93.6 37.6 
92.3 49.4 


51.8*** 
56.0*** 
' 42 . 9*** 


0.000 
0.000 
0.000 


0.025 



SOURCE: MDRC calculations from JOBSTART enrollment form, MIS, and survey data. 



NOTES: Calculations for this table used data for all sample members for whom there were 24 months of , 
follow-up survey data, including those with values of zero for outcomes and those who were assigned to JOBSTART 
but did not participate. 

Average experimental and control group outcomes reported here are adjusted means from two-way 
analysis of covariance procedures controlling for up to 31 kinds of difference in characteristics, other than the 
characteristic used to define subgroups, before random assignment. The two categories used as factors were 
research assignment and, one at a time, the baseline characteristics indicated (see Ostle, 1975, p. 454). There 
may be slight discrepancies in reported sums and differences of these adjusted means because of rounding. 

^A two-tailed t-test was applied to each wi thi n-subgroup impact and also to each difference 
between subgroup impacts. For each characteristic with more than two subgroups, an F-test was applied to the 
interaction between that characteristic and experimental or control status. The columns labeled >" are the 
statistical significance levels of each impact and each difference in impacts or F-statistic: th^.t is, p is the 
probability that sample estimates are non-zero only because of random error. Statistical significance levels are 
indicated as *** « 1 percent; » 5 percent; * = 10 percent. 

^For each characteristic that has only two subgroups, the subgroup impact difference is the impact 
within the first subgroup, less the impact within the second subgroup. 



significant. The ''subgroup impdct*" column shows that this difference in impacts was calculated 
as 55.5 percentage points (the impact for men, shown in the second row) minus 42.1 
percentage points (the impact for women, shown in the third row). 

Other important subgroup comparisons of impacts on service receipt include: 

• Age. Very different patterns of labor market behavior are exhibited at 
each age in the general youth population. Labor force participation, 
employment, and earnings increase dramatically from age 16 to the early 
twenties. Thus, holding everything else constant, youth over age 20 
would pay higher opportunity costs for program attendance than would 
the younger people. The impacts provide some support for this 
generalization, with teenage sample members having a slightly higher 
impact on service receipt (49.7 percentage points) than older youths in 
the sample (44.8 percentage points). 

• Prior employmeiit For those with a more extensive work history, as 
evidenced by employment in the year before random assignment, the 
opportunity costs of participating in a program may be greater. 
Experimentals with and without prior employment had high rates of 
participation in education and training, though the impacts on service 
receipt for those with prior employment were higher because of the 
lower rate of participation in programs by controls. 

• Highest grade attended. While all JOBSTART enrollees were high 
school dropouts, some left school before completing the tenth grade, 
while others dropped out during their junior or senior years. Despite 
the differences in past success in school, the levels of participation and 
impacts on service receipt were nearly identical for the two groups of 
youths. Apparently, JOBSTART sites found ways to engage the lower- 
attainment youths in an education and training program. 

• Welfare receipt Those who receive AFDC, General Assistance, or 
Home Relief may tend to get higher levels of support services such as 
child care, and sometimes may be mandated to participate in some 
program in order to maintain eligibility for their cash benefits. Impacts 
for those not receiving welfare at random assignment were higher than 
for welfare recipients because control group welfare recipients were 
more likely to participate in an education cr training program. 

D. Receipt of Education and Training Separate ly 

Figxire 4.3 reports separately on the monthly receipt of education, showing a large 



^^Since only about 13 percent of the men in the sample received AFDC or General Assistance at 
random assignment, the subgroup receiving welfare was primarily made up of women. 



412- 



FIGURE 4.3 



MONTHLY RECEIPT OF BASIC 
EDUCATION, BY RESEARCH STATUS 
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SOURCE: Appendix Table F.3. 

NOTES: Calculations for this figure used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes and 
those who were assigned to JOBSTART but did not participate. 

"Basic education" includes JOBSTART and non-JOBSTART education activities. 

Random assignment did not always take place on the first of the month. For some 
sample members, the month of random assignment may be less than a month, beginning with the 
date of random assignment and ending on the last day of the month. 
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difference between experimentals and controls in their participation in education throughout 
much of the two-year follow-up period. During the first year, 88 percent of experimentals 
participated in education compared to 17 percent of controls, for a difference in education 
participation of 71 percentage points. In the second year, the participation of exj^erimentals 
dropped sharply as they left the JOBSTART program, while that of controls remained 
approximately what it was during the first year, causing the service receipt difference to decline 
to only 4 percentage points. Analysis of subgroup impacts again indicates a somewhat stronger 
service differential for men than for women, because women controls tended to get more 
education services on their own than did men controls. 

Figure 4.4 shows separately JOBSTARTs impact on the receipt of training. 
JOBSTART achieved less of an experimental-control differential for training alone than for 
training and education together. The smaller impact for training than for education resulted 
from the failure of some JOBSTART sites to achieve transitions into training from education, 
as described in Chapter 3. Similar patterns of impacts on receipt of training appeared for men 
and women, but again the impacts tended to be larger for men. 

II. Impacts on Educational Attainment 

As indicated in the 1989 interim report. Implementing JOBSTART, the impacts of 
JOBSTART on educational attainment during the first year of follow-up were quite similar to 
those of the program that inspired it, the residential Job Corps. An evaluation of the Job 
Corps found that 24 percent of Corpsmembers, but only 5 percent of the comparison group, 
had high school diplomas or GEDs six months after termination from the program (the period 
roughly equivalent to a year of post-random assignment follow-up). -j^j^ report carries the 
story forward to the end of the two-year follow-up period. 

JOBSTART impacts on educational attainment are presented in Table 4.5 for the full 
sample. Table 4.6 presents separate results for men, women living wiih their own children, and 
other women. The severe and intractable problems in reading and mathematics for the young 
adults in JOBSTART are reflected in the low rates of completing high school or passing the 



The two-year impact on receipt of training was 41.6 percentage points for the full sample, 36,0 
percentage points among women, and 49.0 percentage points among men. A comparison with the figures 
for education or training in Table 4.1 supports this point in the text. 
^^Mallar et al., 1982. See also Betsey et al., 1985, p. 112. 
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FIGURE 4.4 



MONTHLY RECEIPT OF TRAINING. 
BY RESEARCH STATUS 
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SOURCE: Appendix Table F.4. 

oP^^^'u Calculations for this figure used data for all sample members for whom there 
were 24 months of follow-up survey data, including those wrth values of zero for outcomes and 
tliose who were assigned to JOBSTART but did not participate. 

Training' i, .:„jdes JOBSTART and non-JOBSTART occupational sl<ills training 
activities. ^ 

Random assignment did not akays take place on the first of the month For ome 
sample members, the month of random assignment may be less than a month, beginning wiiu the 
date of random assignment and ending on tha last day of the month 
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TABLE 4.5 



IMPACTS ON EDUCATIONAL ATTAINMENT 
THROUGH MONTH 24 



Outcome and Fol low-Up Period Experi mentals 


Controls 


Di f f erence 


^a 
P 


Received GEO by end of month 24 


30.7% 


11.9% 


18.9-*** 


0.000 


Received GEO or high school diploma 
by end of month 24 


33.2 


16.4 


16.7*** 


0.000 


Received trade certificate or 
license by end of month 24 


19.2 


8.9 


10.3*** 


0.000 


Received associate's or 2-year college 
degree by end of month 24 


0.2 


0.0 


0.2 


0.146 


Received bachelor's or 4-year 
college degree by end of month 24 


0.0 


0.0 


0.0 


1.000 


Sample size 


949 


890 







SOURCE: MDRC calculations from JOBSTART enrollment fonn and survey data. 

NOTES'. Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up su.vey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted 
means from linear analysis of covariance procedures controlling for up to 31 kinds of 
difference in characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987; 
and Appendix Table B.4). There may be slight discrepancies in reported sums and differences 
of these adjusted means because of rounding. 

=The column labeled "p" is the statistical significance level of the 
difference between experimental and control group outcomes: that ■ , p is the prooability 
that average outcomes are different only because of random error. A two-tailed t-test was 
applied to each difference between average experimental and control group outcomes. 
Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 
percent. 
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TABLE 4.6 



IMPACTS ON EDUCATIONAL ATTAINMENT THROUGH MONTH 24, 
BY GENDER AND PARENTAL STATUS 



Subgroup, Outcome, 
and Follow-Up Period 



Experimental s Controls Difference 



Men 



deceived GED by end of month 24 (%) 28.8 



Received GED or high school 
diploma by end of month 24 (%) 

Received trade certificate or 
license by end of month 24 (%) 

Received associate's or 2-year 
college degree by end of 
month 24 {%) 



32.1 



22.1 



0.0 



Received bachelor's or 4-year 

college degree by end of month 24 (%) 0.0 

Sample size 438 

Uomen living with own child (ren) 

Received GED by end of month 2/" (%) 33.6 

Received GED or high school 

diploma by end of month if4 (%) 35.5 

Received trade certificate or 

license by end of month 24 (%) 16.7 

Received associate's or 2-year 

college degree by end of 

month 24 (%) 0.4 

Received bachelor's or 4-year 

college degree by end of month 24 (%) 0.0 



Sample size 



250 



12.0 
16.9 
9.1 

0.0 

0.0 
433 

10.6 
14.2 
12.1 

0.0 

0.0 
234 



16.8*** 



15.2*** 



12.9" 



0.0 



0.0 



23.0" 



21.3" 



4.6 



0.4 



0.0 



0.000 



0.000 



0.000 



1.000 



1.000 



0.000 



0.000 



0.148 



0.297 



1.000 



(conti nued) 
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TABLE 4.6 (continued) 



Subgroup, Outcomep 
and Fol low-Up Period 



Experimental s Control s Di f f erence 



Uomen nuc living with own 
child(ren), including those 
who did not have any 

Received GEO by end of month 24 (%) 31.1 



Received GEO or high school 
diploma by end of month 24 (%) 

Received trade certificate or 
license by end of month 24 (%) 

Received associate's or 2-year 
college degree by end of 
month 24 (%) 



32.4 



16.2 



0.4 



Received bachelor*s or 4-year 

college degree by end of month 24 (%) 0.0 



Sample size 



261 



13.0 
18.2 
5.7 

0.0 

0.0 

223 



18. r 



14.2^ 



10.6^ 



0.3 



0.0 



0.000 



0,000 



0.000 



0.415 



1.000 



SOURCE: MDP^ calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted 
means from split-file linear analysis of covariance procedures controlling for up to 29 
kinds of difference in characteristics before random assignment (see Ostle, 1975, p. 461; 
Cave, 19C7; and Appendix Table B.4). There may be slight discrepancies in reported sums and 
differences of these adjusted means because of rounding. 

*The column labeled "p" is the statistical significance level of the 
difference between experimental and control group outcomes: that is, p is the probability 
that average outcomes are different only because of random error. A two-tailed t-test was 
applied to each difference between average experimental and control group outcomes. 
Statistical significance levels are indicated i^s *** = 1 percent; ** = 5 percent; * = 10 
percent. 
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GED examination for both experimentals and controls. By month 24 of follow-up, 16.4 
percent of all controls had received a high school diploma or passed the GED examination, 
with about equal proportions among the three subgroups in Table 4.6. Among experimentals, 
33.2 percent had attained a high school diploma or GED, with about equal percentages for 
experimentals among men, women living with their own children, and other women. 

Thus, the full-sample impact on attainment of a GED or high school diploma by month 
24 was 16.7 percentage points, which compares very favorably to the results of the residential 
Job Corps evaluation. The impact was 15.2 percentage points among men, 21.3 percentage 
points among women living with their own children, and 14.2 percentage points among other 
women. 

Compared to the impacts just described for attainment of a diploma or GED, the impact 
on attainment of a GED alone was slightly larger. This was the case because controls (who 
were not as a rule participating in a special alternative program such as JOBSTART) were 
slightly more likely to return to regular high school than were experimentals, although both 
events were rare. The impact on passing of the GED examination was 18.9 percentage points 
for the full sample, 16.8 percentage points among men, 23 percentage points among women 
living with their own children, and 18.1 percentage points among other women. 

These large educational attainment impacts were present for many different subgroups 
in the sample, as shown in Table 4.7 for passing of the GED examination.^"* For example, the 
first three rows of the table present impacts on GED receipt for women and men. For 
women, impacts were 20.8 percentage points, while for men they were 16.8 percentage points. 
The difference in impacts for these two groups, 4 percentage points, is reported in the column 
labeled "subgroup impact difference,** and is not statistically significant. Other subgroup 
findings include a 23.2 percentage point impact on passing of the GED examination for women 
living with their own children, 17.3 percentage points for youths who quit school before the 



^*^e JOBSTART survey also asl .ed about receipt of a trade certificate or license during the follow- 
up period. The precise meaning of these certificates and licenses is not clear. Some could have been 
awarded for completion of a program, rather than for a generally recognized occupational competency. 
Nevertheless, 19.2 percent of experimentals received such certificates and licenses, compared to 8.9 percent 
of controls, for an impact of 103 percentage points. The impact size was slightly higher than this for men 
and about half this for women living with their own children. 

^"^e results presented in this paragraph are bised on an analysis similar to that used for Table 4.4. 
That is, impacts are for subgroups designated by the named characteristic, with differences in other 
observed characteristic? statistically controlled for through linear regression. 
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TABLE 4.7 



IMPACTS ON GEO ATTAINMENT THROUGH MONTH 24, 
BY SELECTED CHARACTERISTICS AT THE TIME OF RANDOM ASSIGNMENT 
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Received GED by 














End of Month 24 






Subgroup 






Sample 






Subgroup 




Impact 




Characteristic and Subgroups 


Size 


Experimental s 


Controls 


Impact 




Di f f erence 


p" 


Gender 












4.0 


0.270 


Women 


968 


33.3V 


12.5V 


20.8*** 


0.000 


W •« V 


__- 


Men 


871 


27.8 


11.0 


16.8*** 


0.000 


a a B 


... 


Ethnicity 












a « « 


0.371 


White, non-Hispanic 


155 


42.5 


24.6 


17.9*** 


0.004 


« W V 




Black, non-Hispanic 


840 


30.1 


11.9 


18.2*** 


0.000 


w « « 


... 


Hispanic 


783 


29.9 


8.9 


21.0*** 


0.000 




... 


Other 


61 


18. 1 


15.1 


3.0 


0.766 


_ _ _ 


... 


Ethnicity, by gender 












— 


0.611 


Women 
















White, non-Hispanic 


84 


49.5 


24.6 


24.9*** 


0.003 






Black, non-Hispanic 


445 


33.2 


12.3 


20.9*** 


0.000 


« 1. « 


. 


Hispanic 


413 


31.6 


10.8 


20.8*** 


0.000 


... 


... 


Other 


26 


13.5 


9.3 


4.2 


0.787 


• - . 


... 


Men 
















White, non-Hispam'c 


71 


33 .8 


24.7 


9.1 


0.329 


... 


... 


Black, non-Hispanic 


395 


26.6 


11.3 


15.2*** 


0.000 


... 


... 


Hisp-nic 


370 


27.9 


6.8 


21.1*** 


0.000 


— 


— 


uiner 




22.2 


17.9 


4.3 


0.746 






Parental status 














0.529 


Women living with own 
















child(ren) 
















No 


484 


32.8 


14.5 


18.3*** 


0.000 






Yes 


484 


33.3 


10.1 


23.2*** 


0.000 






Men who have own child(ren) 
















No 


765 


28.5 


11.5 


17.0*** 


0.000 






Yes 


106 


25.0 


9.7 


15.3** 


0.046 







TABLE 4.7 (continued) 







Received HEO by 














End of Month 24 






Subgroup 






Sample 






Subgroup 




Impact 




Characteristic and Subgroups 


Size 


Experimental s 


Controls 


Impact 




Difference 


b pd 


Employed within past year 












-11. 4*** 


0.002 


No 


870 


27.0% 


14.1% 


12.8*** 


0.000 







Yes 


969 


34.1 


9.8 


24.3*** 


0.000 


_ _ _ 


* M B 


Prior employment, by gender 












* * * 


0.001 


Women employed within 
















past year 
















No 


547 


30.8 


13.2 


17.7*** 


0.000 


— 


— 


Yes 


421 


36.2 


11.4 


24.8*** 


0.000 






Men employed within past year 














No 


323 


21.4 


16.9 


4.6 


0.295 






Yes 


548 


31.8 


8.0 


23.8*** 


0.000 


— 


— 


Left school in grade 11 or 12 












2.7 


0.458 


No 


1,078 


29.9 


9.9 


20.0*** 


0.000 


V a. 


... 


Yes 


761 


31.9 


14.6 


17.3*** 


0.000 






Received occupational training 
















within past year 












-7.9 


0.102 


No 


1,529 


30.2 


12.7 


17.5*** 


0.000 


B B B 


... 


Yes 


310 


33.6 


8.1 


25.4*** 


0.000 








Age 














0.412 


16-19 


1,359 


30.9 


12.9 


18.0*** 


0.000 


... 


... 


20 or 21 


480 


30.2 


8.8 


21.4*** 


0.000 


. B B 


... 


Age, by gender 














0.594 


Women 
















16-19 


710 


33.4 


13.4 


20.1*** 


0.000 





— 


20 or 21 


258 


33.2 


10.4 


22.8*** 


0.000 






Men 
















16-19 


649 


28.0 


12.2 


15.7*** 


0.000 






20 or 21 


222 


26,9 


7.1 


19.8*** 


O.OOn 







ERIC 



(continued) 



TABLF 4.7 (continued) 



I 



Received 6ED by 
End of Month 24 



Subgroup 



Characteristic and Subgroups 


Sample 
Size 


Experimental s 


Controls 


Subgroup 
Impact 




Impact 
Di fference 




Marital status 
Ever married 
Never married 


174 
1,665 


40.7% 
29.7 


6.6% 
12.4 


34.0*** 
17.3*** 


0.000 
0.000 




0.00/ 


Living in own household or 
with hov/ai rl f ri tftnH 

No 

Yes 


1,500 
339 


29.9 
34.2 


11.5 
13.4 


18.5*** 
20.9*** 


0.000 
0.000 


-2 . 4 


0. 608 


Own AFDC pa^p or rprpivinn 
General Assistance 

No 

Yes 


1,344 

495 


29.5 
34.1 


10.3 
16.0 


|Q ^ 2*** 
18.1*** 


0.000 
0.000 


1.1 


0.788 


Own AFDC case 
No 
Yes 


1,446 
393 


31.3 
28.5 


12.4 

9.7 


18.9*** 
18.8*** 


0.000 
0.000 


0.1 


0.984 


Receiving Food Stamps 
No 
Yes 


1,143 

696 


29.5 
32.8 


n.o 

13.2 


18.4*** 
19.6*** 


0.000 
0.000 


-1.2 


0.752 


Arrested since age 16 
No 
Yes 


1,567 
272 


30.6 
31.5 


11-9 
11.3 


18,6*** 
20.1*** 


0.000 
0.000 


1 c: 


n 779 


Lived with both parents at 
age 14 

No 
Yes 


1,198 
541 


31.2 
29.3 


J1.2 
13.0 


20.0*** 
16.8*** 


0.000 
0.000 


3.2 


0.397 



(continued) 
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RptpivpH (iFn hv 
End of Month 24 






Subgroup 

Impact 
Difference^ p^ 


Characteristic and Subgroups 


Sample 
Size 


Experimental s Controls 


Subgroup 
Impact 




Reason for leaving regular 
nign scnooi 

School -rel ated 

Job-related 

Other 


886 
182 
771 


31.0% 12.9% 
29.1 16.0 
30.6 9.6 


18.1*** 

13.1** 

21.0*** 


u.ooo 

0.025 
0.000 


0.440 



SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 



NOTES: Calculations for this table used data for all sample members for whom there were 24 months of 
follow-up survey data, including those with values of zero for outcomes and those who were assigned to JOBSTART 
but did not participate. 

Average experimental and control group outcomes reported here are adjusted means from two-way 
analysis of covariance procedures controlling for up to 31 kinds of difference in characteristics, other than the 
characteristic used to define subgroups, before random assignment. The two categories used as factors were 
research assignment and, one at a time, the baseline characteristics indicated (see Ostle, 1975, p. 454). There 
may be slight discrepancies in reported sums and differences of these adjusted means becau i of rounding. 

^A two-tailed t-test was applied to each wi thin-subgroup impact and also to each difference between 
subgroup impacts. For each characteristic with more than two subgroups, an F-test was applied to the interaction 
between that characteristic ^nd experimental or control status. The columns labeled "p" are the statistical 
significance levels of each impact and each difference in impacts or F-statistic: that is, p is the probability 
that sample estimates are non-zero only because of random error. Statistical significance levels are indicated 
as *** « 1 percent; « 5 percent; * « 10 percent. 

'^For each rharacteri sti c that has only two subgroups, the subgroup impact difference is the impact 
within the first subgroup, less the impact within the second subgroup. 



eleventh grade, and 18.8 percentage points for youths with their own AFDC case at random 
assignment. 
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CHAPTERS 

IMPACTS ON EMPLOYMENT. EARNINGS. AND OTHER OUTCOMES 

Chapter 4 showed that JOBSTART had strong impacts on completion of high school 
or receipt of a GED for most subgroups of the full sample, and that these strong impacts 
remained large over time. This chapter focuses on the extent to which the impact on 
educational attainment has, to this point, translated into increased labor market success and less 
reliance on public beneOt programs. The following key questions about labor market and other 
outcomes are addressed: 

• What sacrifice of employment opportunities or earnings did experimental 
make during the first year after random assignment, which for many was 
primarily a period spent in JOBSTART? 

• Had the experimentals begun to catch up with the controls in 
employment and earnings by the end of the second year? 

• How did experimentals and controls compare in earnings and 
employment within important subgroups? Were the labor market effects 
of JOBSTART different for men, young mothers, and other women? 
Did key JOBSTART impacts vary according to age, grade at time of 
dropout, or other characteristics of the young people in the sample? 

• Do more precise measures of work effort and further information on 
wages earned shed light on the basic employment rate and earnings 
impacts that were observed during the follow-up period? 

• What effect did the program have on other outcomes such as time spent 
in productive activity (that is, employment, education, or training), receipt 
of public benefits, criminal activity, and childbearing? 

The findings described here, based on two years of follow-up, are interim results; they 
represent the early post-program period for most of the sample. The analysis in this chapter 
does not attempt any projections of employment, earnings, or other outcomes into the future. 
The third \/ave of the follow-up survey, fielded at approximately four years after random 
assignment of the sample members, will extend the data for an additional two years. 

A summary of this chapter may prove useful at the outset. The analysis focuses on the 
full sample and three key subgroups: mothers, other women, and men. In brief: The impacts 
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on employment and earnings were encouraging for mothers; small, but slightly positive, for 
other women; and generally negative throughout the period for men. The analysis presents 
impacts on employment and tola! earnings, using simple measures, and then offers possible 
explanations for the basic results. 

In more detail: After an initial period of activity in JOBSTART, experimentals in the 
full sample gradually caught up with controls in employment rate. For young mothers and 
other women, the rate slightly exceeded that of their control group counterparts by the second 
year of follow-up, while men drew approximately even with controls. For the full sample over 
the entire 24-month period, experimentals remained below controls in cumulative earnings, but 
there were important differences among the key subgroups. Young mothers in the 
experimental group earned more than controls during the 24-month period; other women in 
the experimental group earned more than controls during the second 12-month period, nearly 
canceling out their first-year earnings loss. The earnings of male controls substantially 
exceeded those of male experimentals in both years. 

Since earnings are a function of hours and wages, analyzing these more refmed measures 
of labor market success helps explain the basic earnings impact fmdings. Despite the 
employment rates of experimentals and controls being nearly equal during the second year of 
follow-up, controls remained ahead in actual time spent working, although again there were 
differences among the key subgroups. Throughout most of the follow-up period, a higher 
proportion of experimentals than controls were attending education or training classes, which 
may explain, or partly explain, why experimentals spent less time working. In other words, 
investment in human capital continued throughout the follow-up period, supporting the view 
that the final story on employment impacts cannot yet be told. 

L JOBSTART as an Investment 

If programs like JOBSTART are effective, they will lead to long-run gains in 
employment and earnings, but in the short run there may be negative impacts. During the first 
year after entering the sample, JOBSTART youths had less time available for work than did 
controls, since for much of that period, they were in an intensive program of education and 
skills instruction. Chapter 3 showed that more than 50 percent of experimentals were still 
active in the sixth month after entering the sample; 16 percent were still active in the twelfth 
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month; and almost 10 percent were still active in the fifteenth month. Throughout most of the 
two-year follow-up period, a larger percentage of experimentals than controls were participating 
in a program. Controls, therefore, got a head start in the labor market, since experimentals 
could not be in JOBSTART and be working during any given hour of the day. Also, for 
young people with poor skills, work experience itself can be an important source of new job 
skills and higher wages. 

GED attainment during or after ntensive JOBSTART education ultimately may open 
up many employment opportunities for JOBSTART graduates. But even after a sample 
member left JOBSTART, it might still take some time to become as well-settled in the labor 
market as his or her control group counterpart, who might have been learning new skills while 
working. Thus, well into the second year of follow-up, controls might still be expected to have 
had the edge on experimentals in employment and earnings. 

The evaluation of the residential Job Corps provides an example of this point. ^ That 
study began its follow-up at the point of termination from the program, and it reported impacts 
only for sbc-month intervals rather than for individual months. For the period from program 
termination to six months thereafter, the employment and earnings of those in the program 
group were slightly lower than those of the comparison group (although the difference was not 
statistically significant). The employment and earnings rates of the program group began to 
exceed those of the comparison group six to 12 months following termination from the 
program, and they continued to be higher throughout the remainder of the 48-month follow- 
up period. 

These findings cannot be applied directly to the JOBSTART evaluation because follow- 
up in JOBSTART began at random assignment - the point when youths were ready to enter 
the program. However, since the average length of stay in JOBSTART was roughly six 
months, the 12-month survey was conductea about six months after the average sample 
member left the program, and the 24-month survey was conducted about 18 months after exit 
from the program. In sum, the monthly employment and earnings impacts in the Job Corps 
study turned positive during a period roughly corresponding to the second year of follow-up 
covered by this report. 



^See Mallar et al, 1982, p. 135. 
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IL JOBSTARTs Impacts on Employment Rates 

Because young adults tend to apply for employment and training programs when they 
are between jobs, the employment rate grew over time for both experimentals and controls 
(Table 5.1). For the full sample, a larger fraction of controls than experimentals was employed 
in each month of the first year.^ The difference in employment rate peaked in month five and 
then mainly declined over time. After month 12, the employment rate of experimentals was 
greater than that of controls more oltcn than it was below it, but in no month was this 
difference statistically significant. Abc^ut 46 percent of each group worked at some point 
during the last month of follow-up (month 24). During the second year as a whole, 69.5 
percent of controls worked at some point, did 72.0 percent of experimentals, for a positive 
employment rate impact of 2.5 percentage points. This impact was, however, not statistically 
significant. 

A. Impacts by Gender 

As explained in Chapter 4, many previous evaluations of youth employment and training 
programs have found better program effects for women than for men.^ One likely explanation 
for women's better employment results is that it is easier to improve the employment and 
earnings of those who do not spend much time in the world of work (for example, young 
mothers) than of those who are already in the labor force but fail to find and keep steady, 
well-paying jobs (for example, young men with poor skills). Thus, from this perspective, 
women have greater potential for improved labor market outcomes than do men, and less to 
lose (in terms of forgone employment and earnings) by investing :n education and training. 

The JOBSTART sample exemplifies this pattern. During the first year after random 



appearance of a sharp increase in employment rates in month 13 is very likely due to survey 
measurement error. As explained in Appendix A 235 of the 1,839 24-month survey completers responded 
to a single combination instrument covering their labor market behavior over the entire two-year period, 
while 1,604 of them responded to separate survey efiforts 12 and 24 month^ aftei random assignment. 
Separate analysis of the 235 "combination* responders showed continuous gro>^ \ en:ployment rates over 
the 24-month follow-up period. But many in the sample who responded to a su. y after 12 and 24 months 
showed a discontinuity in employment rates at month 13 owing to poor and inconstnent recall. For 
example, respondents to the 24-month survey reported employment at the start of the period covered by 
the survey (month 13) that probably started somewhat later. This happened even though ^he survey staff 
prompted respondents with their previous responses to questions about employment in moiuh 12. 
^ee Betsey et al., 1985, for a suivey. 

-128- 




TABLE 5.1 



IMPACTS ON EMPLOYMENT RATES 
THROUGH MONTH 24 



Outcome and Follow-Up Period Experimental s Controls Difference p* 



Ever employed 



Months 1-24 


79.2% 


77 9% 


1 1 




Months 1-12 


57 .6 


61 6 


-4 0* 




Months 13-24 


72.0 


69. 5 


2 5 




Quarter 1 


20.5 


2Q 8 




n nnn 


Quarter 2 


30,4 


40. 1 




n nnn 


Quarter 3 


40.7 


44 2 


•J.J 


n 111 


Quarter 4 


48 1 


49 6 


• 1 • J 


n cm 


Quarter 5 


54.6 


52 7 


1 ft 


n 107 


Quarter 6 


48.1 


47 9 


0 ? 


n QAn 

U . !7HU 


Quarter 7 


50.5 


50.6 


-0 1 


n Q^d 

u . :y jh 


Quarter 8 


53.4 


51.7 


1 6 


n dm 

U . H J J 


ter employed 










Month 1 


13.1 


20.4 


-7 1*** 
f . J 


n nnn 

U . UUv 


Month 2 


16.5 


24.4 




n nnn 

u • uuu 


Month 3 


18.5 


27.0 


-8.5*'** 


0 nnn 

w • uuu 


Month 4 


ZlA 


30.3 


7 * c 


n nnn 

u • uuu 


Month 5 


23.6 


33.1 


-9 5*** 


0 000 
\j » www 


Month 6 


26,5 


34.0 


-7 ,4*** 


0 nnn 

\j . uuu 


Month 7 


30.3 


35.7 


-5.3** 


n nil 

u * U 1 1 


Month 8 


32.2 


37.2 


-5.0** 


n niQ 

U * U A 7 


Month 9 


35.3 


38.3 


-3.0 


0 159 


Month 10 


38.5 


39.6 


-1.2 


0 592 

W.J 7C 


Month 11 


38.1 


41.8 


-3.7* 


0.089 


Month 12 


41.0 


42.2 


-1.3 


0.562 


Month 13 


47.7 


45.4 


2.4 


0.281 


Month 14 


39.2 


36.4 


2.8 


0.187 


Month 15 


41.2 


39.3 


1.9 


0.385 


Month 16 


41.6 


40.0 


1.6 


0.461 


Month 17 


41.2 


42.1 


-0.9 


0.684 


Month 18 


42.3 


43.3 


-1.0 


0.645 


Month 19 


42.3 


44.1 


-1.8 


0.403 


Month 20 


42.9 


43.7 


-0.8 


0.717 


Month 21 


44.8 


44.0 


0.8 


0.726 


Month 22 


46.2 


44.5 


1.7 


0.441 


Month 23 


45.6 


45.8 


-0.2 


0.941 


Month 24 


46.1 


46.7 


-0.7 


0.763 



Sample size 949 890 



SOURCEt MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

(continued) 
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TABLE 5.1 (continued) 



Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here ^re adjusted 
means from linear analysis of covariance procedures controlling tor up to 31 kinds of 
difference in characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987; 
and Appendix Table B.4). There may be slight discrepancies in reported sums and differences 
of these adjusted means because of rounding. 

*The column labeled "p" is the statistical significance level of the 
difference between experimental and control group outcomes: that is, p is the probability 
that average outcomes are different only because of random error. A two-tailed t-test was 
applied to each difference between average experimental and control group outcomes. 
Statistical significance levels are indicated as = 1 percent; ** = 5 percent; * = 10 
percent. 
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assignment, 74.8 percent of control men worked, compared to 38.3 percent of women living 
with their own children and 61.5 percent of other women (Table 5.2).^ During this period, 
which, for many experimentals, was largely a time of program participation, the impact of 
JOBSTART on employment rates was 2.7 percentage points for the young mothers, -4.5 
percentage points for other women, and -8.1 percentage points for men. Thus, JOBSTART 
entailed an opportunity cost (forgone employment) tor men and women who were not caring 
for children, but not for young mothers. 

A second possible explanation for the poorer impacts observed for men in JOBSTART 
is the greater difficulty of placing men in jobs that value a GED. Many women in 
JOBSTART, for example, were trained in clerical occupations and sought clerical jobs, for 
which educational credentials were important. However, young men were more likely to train 
for work in occupations that did not, at least initially, value a GED - for example, in many 
types of blue-collar work, especially physically demanding jobs.^ Further, it may have been 
harder to find training-related jobs for men who did study for occupations in which having a 
GED matters. These explanations for women's stronger impacts will be explored in future 
work on the JOBSTART Demonstration. So far (during two years of follow-up), the 
investment in JOBSTART services follows the usual pattern and has not paid off for men in 
higher employment rates. Although they regained the ground they lost in the first year, they 
did not surpass the controls in the second year. Table 5.2 shows that the impact on 
employment rates was precisely zero for men during the second year of follow-up; exactly 84.3 
percent of male experimentals and controls worked at some point during those 12 months. 

In contrast, female experimentals especially young mothers - seem to have overtaken 
female controls in employment during the same period. Table 5.2 shows that the impact on 
employment rates for young mothers was 7.2 percentage points during months 13 through 24, 
and for other women it was 4.2 percentage points; these impacts were not, however, 
statistically significant. 

The month-by-month story is less clear than that told by cumulative yearly employment 
rates. Figure 5.1 shows monthly employment rates for experimentals and controls among men, 



"in the rest of this analysis, women living with their own children will be referred to as "mothers" 
even though some of the "other women" may be mothers who were not living with their children. 

^Chapter 3 reported on the occupations for which men and women trained and showed a much higher 
percentage of women rtudying for clerical occupations. 
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TABLE 5.2 



IMPACTS ON EMPLOYMENT RATES THROUGH MONTH 24, 
BY GENDER AND PARENTAL STATUS 



Subgroup, Outcome, and Follow-Up Period Experimental s Controls Difference p* 



Hen 



Ever employed (%) 
Months 1-24 
Months 1-12 
Months 13-24 

Sample size 

yomen living with own child (ren) 

Ever employed (%) 
Months 1-24 
Months 1-12 
Months 13-24 

Sample size 

Uomen not living with own child (ren), 
including those who did not have any 

Ever employed (%) 
Months 1-24 
Months 1-12 
Months 13-24 

Sample size 



89.0 


90.1 


-1.1 


0.583 


66.7 


74.8 


-8.1*** 


0.006 


84.3 


84.3 


0.0 


0.990 


438 


433 

♦ 






62.1 


57.2 


4.9 


0.279 


41 .0 


38.3 


2.7 


0.543 


53.3 


46.1 


7.2 


0.113 


250 


234 






79.3 


75.8 


3.4 


0.363 


57.0 


61.5 


-4.5 


0.311 


69.5 


65.3 


4.2 


0.332 


261 


223 







SOURCES FOR 'iABLE 5.2 AND FIGURE 5.1: MDRC calculations from JOBSTART enrollment form 
and survey data (Table 5.2); Appendix Table F.5 (Figure 5.1). 

NOTES FOR TABLE 5.2 AND FIGURE 5.1: Calculations for this table and figure used 
data for all sample members for whom there were 24 months of follow-i'p survey data, 
including those with values of zero for outcomes and those who were assigned to JOBSTART but 
did not participate. 

Random assignment did not always take place on ths first of the month. For some sample 
members, the month of random assignment may be less than a month, beginning with the date of 
random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means from 
split-file linear analysis of covariance procedures controlling for up to 29 kinds of 
difference in characteristics before random assignment (see OsMe. 1975, p. 461; Cave, 1987- 
and Appendix Table 8.4). There may be slight discrepancies in reported sums and differences 
of these adjusted means because of rounding. 

*The column labeled "p" is the statistical significance level of the difference between 
experimental and control group outcomes: that is, p is the probability that average 
outcomes are different only because of random error. A two-tailed t-test was applied to 
each difference between average experimental and control group outcomes. Statistical 
significance levels are indicated as *** « 1 percent; ** « 5 percent; * = 10 percent. 
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FIGURE 5.1 



MONTHLY EMPLOYMENT RATES, 
BY GENDER AND PARENTAL STATUS 
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young mothers, and other women. Young mothers who were in the experimental group appear 
to have caught up with, and sometimes overtaken, their control group counterparts in monthly 
employment rates part-way through the first year, while other women in tAe experimental 
group caught up with their control group counterparts at about the beginning of the second 
year. For the **other women"* subgroup, there also appears to have been a trend toward 
growing positive impacts in the last six months of the follow-up period. In contrast, 
experimental men remained slightly behind control men during most of the second year. Thus, 
although the jury is still out on employment rate impacts, the findings certainly are more 
encouraging for women than for men. 

B. Impacts for Other Subgroups 

In addition to looking at impacts for subgroups separately (for example, looking at the 
impacts for men, women living with their own children, and other women), it is useful to 
estimate employment rate impacts for subgroups controlling for differences in characteristics 
other than the one used to designate the groups (for example, gender).^ lliis helps explain 
whether any observed differences in impacts for n^en, women living with their own children, 
and other women were due to other factors correlated with gender and parenting status (such 
as past employment experience) or continue to be present even after such differences are 
controlled for in the analysis. This type of analysis also aWovfs calculation of whether 
differences between impacts for women and men were statistically significant. 

Table 5.3 addresses these issue; by presenting impacts on the percentage of the sample 
ever employed during months 13 through 24. Each section of the table (such as the first for 
women and men) presents impacts for the designated subgroups, along with the 
between-subgroups impact difference.^ The entry to the right of the first line of the tabie 



^Chapter 2 highlighted ways in which JOBSTART women were different from JOBSTART men in 
characteristics besides gender. The impact estimates for men, young mothers, and other women, in Table 
5.2 were done by splitting the entire sample into three subsamples. To make the estimates more precise, 
the separate estimates for eaca group took into account small obsen^ed differences between experimentals 
and controls in each subsample through regression adjustments. But there were many differences among 
the three subsamples in JOBSTART other than their gender and parenting status; for example, women - 
especially young mothers - had less work experience than did men. This makes it difficult to understand 
the sources of differences in impacts among the three groups. 

^The impact estimates in this table were produced by conducting a two-way analysis of covariance 
controlling for differences in pre-random assignment characteristics other than the characteristic used to 

(continued...) 
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TABLE 5.3 



IMPACTS ON YEAR-TWO EMPLOYMENT RATES, 
BY SELECTED CHARACTERISTICS AT THE TIME OF RANDOM ASSIGNMENT 



Lnorou t en s 1 1 c ono jUD9roups 


Sample 
Si ze 


Ever Employed. 
Experimental s 


Months 13- 
Controls 


24 

Subgroup 
Impact 




SubarouD 

%r U 1^ i V U m0 

Impact 
Duference" 




Gender 












4.9 


0.222 


Women 


968 


64.7% 


60.0V 


4.8* 


0.081 


— 


--- 


Men 


871 


80.0 


80.1 


-0.1 


0 977 






Ethnicity 












— 


0.203 


White, non-Hispanic 


155 


82.9 


78.8 


4.1 


0.550 


— 


— 


nil 1 1 * * 

Black, non-Hispamc 


840 


69.3 


64.1 


5.3* 


0.073 


— 


— 


Hispanic 


783 


73.9 


73.2 


0.7 


0.820 


— 


— 


Other 


61 


55.3 


72.8 


-17.6 


0.110 


— 


— 


Ethnicity, by gender 












— 


0.382 


Women 














White, non-Hispanic 


84 


78.4 


70.1 


8.3 


0.369 


— 


— 


Black, non-Hispanic 


445 


64.0 


54.2 


9.8** 


0.015 


— 


— 


Hispanic 


413 


64.4 


64.6 


-0.3 


0 950 






Other 


26 


40.4 


55.8 


-15.4 


0.364 


— 


«... 


Men 
















White, non-Hispanic 


71 


87.2 


88.6 


-1.4 


0.893 


— 


— 


Black, non-Hispanic 


395 


74.7 


74.7 


-0.0 


0.995 


— 


— 


Hispanic 


370 


84.7 


82.9 


1.7 


0.692 


— 


— 


Other 


35 


70.8 


87.5 


-16.7 


0.255 


— 


— 


Parental status 














0.660 


Women living with own 
















child(ren) 
















No 


484 


67.0 


63.0 


4.0 


0.309 






Yes 


484 


58.5 


52.9 


5.6 


0.148 






Men who have own child(ren) 
















No 


765 


82.7 


82.6 


0.1 


0.986 






Yes 


106 


79.2 


80.3 


-1.1 


0.895 







(continued) 
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TABLE 5.3 (continued) 



Characteristic and Subgroups 


Sample 
Size 


Ever Employed. 
Exoerimental 


Months 13-24 

^ pAfit ml Q 


Subgroup 


P 


Subgroup 
Impact 
ui TTcrence 


„a 
P 


Employed within past year 












3.2 


0.424 


No 


870 


70.0% 




4.2 


0.150 


... 


... 


Yes 


969 


73.8 


:.8 


1.0 


0.725 


... 


... 


Prior employment, by gender 
















Women employed wUhin 












... 


0.414 


past year 
















No 


547 


fii 1 


C7 n 

D/ .U 


A 1 

4. 1 


0.258 






Yes 


421 


68.2 


62.6 


5.6 


0.182 


... 




Men employed within past year 














No 


323 


8C.3 


76.0 


4.2 


0.377 


--- 


— 


Yes 


548 


nn 7 


Ol A 
OO • H 


-d.O 


0.470 






Left school in grade 11 or 12 












-3.0 


0.454 


No 


1,078 


68.9 


67.7 


1.2 


0.637 


... 


... 


Yes 


761 


76.3 


72.1 


4.3 


0.169 





— 


Received occupational training 
















within past year 












6.1 


0.247 


No 


1.529 


72.3 


68.7 


3.5 


0.104 


— 


— 


Yes 


310 




/ 0 tC 


-c.o 


0.592 






Age 












0.0 


0.999 


16-19 


1.359 


72.7 


70.3 


2.5 


0.285 


... 




20 or 21 


480 


69.9 


67.4 


2.5 


0.527 




... 


Age, by gender 












... 


0.681 


Women 














16-19 


710 


65.5 


60.5 


5.0 


0.122 






20 or 21 


258 


62.9 


58.5 


4.3 


0.412 






Men 
















16-19 


649 


80.8 


81.0 


-0.2 


0.952 






20 or 21 


222 


77.7 


77.4 


0.3 


0.955 
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TABLE 5.3 (continued) 



Characteristic and Subgroups 


Sample 
Size 


Ever Emoloved. Months 13-24 
Experimental Controls 


Subgroup 
Impact 




Subgroup 

Impact 
Difference'' 




Marital status 
Ever married 
Never married 


174 
1.665 


77.6V 
71.4 


75.3% 
68.9 


2.2 
2.5 


0.729 
0.231 


-0.3 


0.969 


Living in own household or 
with boy/girlfriend 

No 

Yes 


1,500 
339 


72,2 
71.1 


69.9 
67.9 


2.3 
3.1 


0.295 
0.496 


-0.8 


0.869 


Own AFOC case or receiving 
General Assistance 

No 

Yes 


1,344 
495 


74.1 
66.3 


70.2 
67.7 


3.9* 
-1.5 


0.092 
0.700 


5.4 


0.229 


Own AFDC case 
No 
Yes 


1.446 
393 


73.0 
68,1 


70.3 
66.6 


2.7 
1.5 


0.224 
0.717 


1.2 


0.810 


Receiving Food Stamps 
No 
Yes 


1.143 

69G 


73.3 
6<».9 


68.9 
70.5 


4.3* 
-0.6 


0.085 
0.859 


4.9 


0.231 


Arrested since age 16 
No 
Yes 


1.567 
272 


72.1 
71.2 


70.9 
61.6 


1.2 

!J.6* 


0.569 
0.064 


-8.4 


0.136 


Lived with both parents at 
age 14 

No 

Yes 


1,198 
641 


71.2 
73.4 


66.8 
74.6 


4.5* 
-1.2 


0.069 
0.710 


5.7 


0.170 


Reason for leaving regular 
high school 

School -related 

Job-related 

Other 


886 
182 
771 


74.0 
73.8 
69.1 


69.1 
77.3 
67.9 


4.9* 
-3.5 
1.2 


0.085 
0.585 
0.702 




0.411 
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SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all sample members for whom there were 24 months of 
follow-up survey data, including those with values of zero for outcomes and those who were assigned to JOBSTART 
but did not participate. 

Average experimental and control group outcomes reported here are adjusted means from two-way 
analysis of covariance procedures controlling for up to 31 kinds of difference in characteristics, other than the 
characteristic used to define subgroups, before random assignment. The two categories used as factors were 
research assignment and, one at a time, the baseline characteristics indicated (see Ostle, 1975, p. 454). There 
may be slight^discrepancies in reported sums and differences of these adjusted means because of rounding. 

A two-tailed t-test was applied to each wi thin-subgroup impact and also to each difference between 
subgroup impacts. For each characteristic with more than two subgroups, an F-test was applied to the interaction 
between that characteristic and experimental or control status. The columns labeled -p" are the statistical 
significance levels of each impact and each difference in impacts or F-statistic: that is, p is the probability 
that sample estimates are non-zero only because of random error. Statistical significance levels are indicated 
as *** = 1 percent; ** « 5 percent; * « 10 percent. 

For each characteristic that has only two subgroups, the subgroup impact difference is the impact 
within the first subgroup, less the impact within the second subgroup. 



^ 1 



shows that - controlling for other measured differences in the subgroups - the impact for 
women was 4.9 percentage points greater than the impact for men and that this ^'subgroup 
impact difference*' was not statistically significant.® The "subgroup impact" column shows that 
this difference is calculated as 4.8 percentage points (the impact for the women) less -0.1 
percentage points (the impact for men). The within-subgroup impact was statistically significant 
for women but not for men.^ The more detailed categories based on parenting status by 
gender also show stronger impacts for women than for men, and for mothers than for other 
women, although here again the differences in impacts were not statistically significant. Thus, 
the earlier apparent differences in employment rate impacts for men, young mothers, and other 
women did not disappear (although they were not statistically significant) when other measured 
characteristics of each group were accounted for in the analysis. 

The rest of Table 5.3 presents other subgroup impacts on second-year employment rates 
in a similar fashion. Nothing in the final column is significant in a statistical sense, although 
impact differences seem sizable in absolute terms (more than 5 percentage points) for arrest 
history, prior training, two-parent upbringing, welfare receipt, and ethnicity. 

There is no clear pattern to these differences in impacts. 

• In some of these cases, those with greater barriers to employment 
(represented by control group employment rates) had greater impacts. 
Women had larger impacts than men, youths with no previous training 
had larger impacts than those with previous training, those arrested since 



'(...continued) 

define the subgroup under review. See Ostle, 1975, p. 461. The adjustments were done using a linear 
regression model. Characteristics that affect outcomes and impacts through a nonlinear relationship are 
not controlled for with this procedure. And no such procedure could control for unmeasured characteristics 
that affected outcomes and impacts. The adjusted outcomes and impacts presented here for women and 
for men are not identical to those presented in Table 5.2 because the former attempt to take observed 
non-gender differences between women and men into account. The differences in impacts are very minor, 
but the adjustments change some outcome measures several percentage points. 

^is is indicated by the absence of stars on the "subgroup impact difference." 

^is subgroup impact can be statistically significant, even though it is smaller than the impact for 
youne mothers alone, because the sample size is larger. 

^^As discussed in Chapter 2 and Appendix B, subgroup impact differences must be interpreted carefully 
because characteristics such as gender were not assigned at random and so can be correlated with other 
characteristics. It could very well be that it was not gender per se that explained apparently higher impacts 
for women than for men, but some observed or unobserved non-gender difference between the women and 
the men in this sample. The calculations in Table 5.4 incorporate simple adjustments for observed 
characteristics besides gender, but not all differences were observed, and observed characteristics may have 
had more complex effects on outcomes and impacts than those modeled in these adjustments. 
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age 16 had larger impacts than those who had not been arrested, and 
those who did not live with two parents at age 14 had larger impacts 
than those living in a two-parent family. 

• For other subgroups, however, the opposite pattern emerged. Impacts 
were higher for those not receiving public assistance at random 
assignment than for recipients. 

• For some subgroups - most notably those based on ethnicity - no clear 
pattern emerged. Blacks had the highest impacts, Hispanics had virtually 
no impacts, and "other nonwhites" (primarily Asians) had large negative 
impacts. 

This mixed finding differs from the results of sevoral studies of employment programs for 
welfare recipients, which did find patterns among subgroup impacts. In these programs, 
impacts tended to be small or nonexistent for the most job-ready and the least job-ready and 
positive for a group in the middle tier.^^ Clearly, the subgroups used in the JOBSTART 
analysis are much more narrowly defined, which may contribute to the absence of a pattern in 
the impacts. 

Other characteristics seem to have been more weakly associated with impact differences 
in second-year employment rates: 

• Recent employment Employment at some point in the year before 
enrollment is associated with a 3.2 percentage point lower impact, weakly 
supporting the position that it is easier to raise employment rates for 
those who have less of a work history. 

• Recent employment, by gender. Employment rate impacts were positive 
both for women who had some employment and for women who had no 
employment in the year before program entry. However, for men, the 
positive impacts for those who were not employed were canceled out 
by negative impacts for those who had some employment in the 
pre-program year. 



"See Friedlander, 1988. 

second reason for not highlighting differences in impacts for subgroups is that the findings are 
somewhat sensitive to the outcome measure chosen. For example, when a similar analysis is done for 
employment in the last three months of follow-up, the results are somewhat different, but the general 
conclusion (that there is no clear pattern to the subgroup findings) remains. 

^^For the previously employed women, the impact in year two was 1.5 percentage points higher than 
the impact for women who were not employed, although this subgroup impact difference was not statistically 
significant. 
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• Highest grade attended. While all JOBSTART enroUees were high 
school dropouts, some left school in the ninth grade, while others 
dropped out during their senior year Although the difference in impacts 
was not statistically significant, second-year employment was improved by 
a wider margin for those vho dropped out in later grades than for Ihose 
who dropped out earlier.^* 

• Recent prior skills training. For a substantial minority of the sample, 
JOBSTART was not the Grst try at a second-chance program. The 
second-year employment impact was 6.1 percentage points higher (not 
statistically significant) for those who had not tried another program 
during the 12 months preceding enrollment in JOBSTART. 

• Age. Very different patterns of labor market behavior are shown at each 
age in the general youth population. Labor force participation, 
employment, and earnings increase dramatically from age 16 to the early 
twenties. Moreover, program operators often suggest that younger 
enroUees do not derive as much benefit from training as do somewhat 
older youths who are ready to "settle down" and pursue stable 
employment. Table 5.3 gives no support whatever to this hypothesis: 
The difference in impacts between teenagers and older youths is precisdy 
zero and the younger youths have higher employment rates. 



III. JOBSTARrs Impacts on Earnings 

The full-sample impact of JOBSTART on earnings during the two-year follow-up period 
is less encouraging than on employment rates, although once again there is some evidence of 
movement in the right direction. As shown in Table 5.4, during the first 12 months following 
random assignment, controls earned $585 more than experimental, while during the second 12 
months, controls earned $205 more. Once again, the month-by-month pattern of impacts is 
less clear than the pattern of annual impacts: While experimentals moved even with controls 
during months 14 and 15, after that point the earnings impact fluctuated within a fairly narrow 
and slightly negative range. 



difference was driven by lower employment for controls who dropped out earlier rather than by 
higher employment for experimentals who dropped out earlier. 

^^On this point, analysis of the employment rate for the last t ee months of follow-up suggests a 
different result: Younger women do better than older women, while older men do better than younger 
men. 
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TABLE 5.4 



IMPACTS ON EARNINGS THROUGH MONTH 24 



Outcome and Fol low-Up Period Experimental s Controls Difference p* 



Total earnings 



Months 1-24 


$5,859.56 


$6 649.07 


-$789. b2*** 


0.010 


Months 1-12 


1.965.47 


2.550,26 

mm A W W V 9 mm w 


-584.78*** 


0.000 


Months 13-24 


3.894,08 


4.098,81 


-204.73 


0.3"?1 

V • V X 


Fami nas 










Month 1 


37 ,92 


64,30 


-26.39*** 


0 001 

V , W X 


Month 2 


93 65 


141 78 


-48 13*** 


0 000 


Month 3 


109 16 


165 51 


-5S '55*** 


0 000 


Month 4 


41 


187 20 


V X • / 7 


0 000 


Month 5 


146 31 


211 13 


-64 81*** 


0 000 


Month 6 




222 28 


-64 22*** 


0 000 


Month 7 


179 79 


234 87 


-55 08*** 


0 001 


Month 8 


196 91 


240 63 


-43 72** 

t t Cm 


0 01 1 


Month 9 


213 35 


256 66 


-43.31** 


0 01? 


Mon*h 10 


22ft 26 


265 40 


-37 15** 


0 0?fi 


Month 11 


226.07 


281.29 


-55.22*** 


0.002 


Month 12 


250.59 


279.20 


-28.61 


0.115 


Month 13 


254.54 


273.39 


-18.85 


0.289 


Month 14 


294.66 


291.32 


3.33 


0.870 


Month 15 


313.56 


309.39 


4.17 


0.839 


Month 16 


314.47 


324.95 


-10.48 


0.609 


Month 17 


317.17 


340.33 


-23.16 


0.269 


Month 18 


322.40 


346.92 


-24.52 


0.239 


Month 19 


320.80 


364.48 


-43.68** 


0.041 


Month 20 


336.01 


357.61 


-21.60 


0.320 


Month 21 


351.86 


360.39 


"8.53 


0.695 


Month 22 


354.24 


370.68 


-16.^4 


0.445 


Month 23 


355.78 


373.41 


-17.63 


0.417 


Month 24 


358.60 


385.95 


-27.35 


0.214 



Sample size 949 890 



SOURCE: MDRC calculations from JOBSTART enrollment fonn and survey data. 

NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month, 
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TABLE 5.4 (continued) 

Average experimental and control group outcomes reported here are adjusted 
means from linear analysis of covanance procedures controlling for up to 31 kinds of 
difference in characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987; 
and Appendix Table B.4). There may be slight discrepancies in reported sums and differences 
of these adjusted means because of rounding. 

*The column labeled "p" is the statistical significance level of the 
difference between experimental and control group outcomes: that is, p is the probability 
that average outcomes are different only because of random error. A two-tailed t-test was 
applied to each difference between average experimental and control group outcomes. 
Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 
percent. 



V.Ul 



A. Impacts bv Gender 

As with employment rates, the impacts for earnings at this point seem more encouraging 
for women than for men. Table 5.5 shows that for men, controls earned $1,028 more than 
experimentals in the Grst year of follow-up and $667 more in the second. For women who 
were living with their own children, experimentals have already pulled slightly ahead of 
controls; experimentals earned $89 less in the first year and then earned $172 more in the 
second year. The trend for other women is also in a positive direction: An earnings loss of 
$423 was followed by an earnings gain of $202. 

B. Impacts for Other Subgroups 

Table 5.6 presents second-year earnings impacts by selected subgroup and differences 
among these subgroup impacts.^^ As in Table 5.3, this table presents comparisons of subgroup 
impacts for the designated groups, 'jontrolling for differences other than the one i«ed to define 
the subgroup categories. The fi'.'st line of the table shows that the difference in second-year 
earnings impact by gender (controlling for other measured differences of men and women) 
was $826 (-$639 for men versus $187 for women) and that it was statistically significant. The 
observed male-female earnings differences in impacts, presented in Table 5.5, therefore, do not 
appear to have resulted from other measured non-gender differenc^^ in their pre-random 
assignment characteristics (such as differences in education or work experience.) 

The rest of Table 5.6 shows impacts by other subgroup splits. The results generally 
mirror what already has been said about employment rates, with a few exceptions. 

IV, Moving Behind These Emplovmcnl-Related Impacts 

The analysis presented so far can be summarized as showing that employment rates for 
experimentals have caught up with those of controls during the follow-up period, but 
(especially for men) earnings of experimentals still lag behind. A starting point for 
understanding this result is to look behind the summary measure of employment success 



^^Male controls also out-earned experimentals in every month of the follow-up period, though during 
much of the second yeir, these differences were not statistically significant. 

^^As in Table 5.4, adjusted outcomes and impacts by gender differ slightly from their counterparts in 
previous tables because of the different way they were calculated, using the entire sample rather than 
splitting by gender and making separate calculations. 
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TABLE 5.5 



IMPACTS ON EARNINGS THROUGH MONTH 24, 
BY GENDER AND PARENTAL STATUS 



Subgrcupt Outcome, 
and Fol low-Up Period 



Experimentals Controls 



Difference 



Hen 



Total earnings ($) 
2-year total 
Year 1 
Year 2 
Quarter 1 
Quarter 2 
Quarter 3 
Quarter 4 
Quarter 5 
Quarter 6 
Quarter 7 
Quarter 8 

Sample size 

Uomen living with own child (ren) 

Total earnings ($) 
2-year total 
Year 1 
Year 2 
Quarter 1 
Quarter 2 
Quarter 3 
Quarter 4 
Quarter 5 
Quarter 6 
Quarter 7 
Quarter 8 

Sample size 

Uomen not living with own child(ren), 
including those who did not have any 

Total earnings ($) 
2-year total 
Year 1 
Year 2 
Quarter 1 
Quarter 2 
Quarter 3 
Quarter 4 
Quarter 5 
Quarter 6 
Quarter 7 
Quarter 8 

Sample size 



7,797.22 


9,492.61 


-1,695.39*** 


0.001 


2,648.50 


3,676.88 


-1,028.38*** 


0.000 


5,148.72 


5,815.73 


-667.01* 


0.057 


348.17 


551.11 


-202.93*** 


0.001 


581.48 


918.15 


-336.67*** 


0.000 


796.52 


1,022.11 


-225.60*** 


0.008 


922.33 


1,185.50 


-263.18*** 


0.003 


1,119.86 


1,286.81 


-166.94* 


0.075 


1,266.70 


1,426.00 


-159.30 


0.112 


1,354.56 


1,506.69 


-152.13 


0.134 


1,407.60 


1,596.24 


-188.64* 


0.063 


438 


433 







3,035.55 
1,011.83 
2,023.73 
75.56 
200.41 
336.41 
399.44 
474.35 
533.50 
543.85 
472.03 

250 



2,952.31 
1,100.96 
1,851.35 
151.45 
247.18 
328.49 
373.84 
355.07 
447.95 
503.85 
544.47 

234 



83.24 
-89.13 
172.38 
-75.89** 
-46.77 
7.93 

2C.60 
119.27 

85.55 

40.00 
-72.45 



0.852 
0.647 
0.600 
0.036 
0.396 
0.905 
0.733 
0.150 
0.361 
0.682 
0.461 



5,070.23 


5,290.80 


-220.58 


0.667 


1,607.50 


2,030.02 


-422.53* 


0.055 


3,462.73 


3,260.78 


201.95 


0.603 


191.97 


285.26 


-93.29** 


0.045 


353.67 


482.98 


-129.31* 


0.06: 


465.73 


617.07 


-151.34* 


0.052 


596.12 


644.71 


-48.59 


0.540 


766.80 


660.17 


106.63 


0.254 


796.34 


842.71 


-46.36 


0.662 


860.12 


881.52 


-21.39 


0.850 


1,039.45 


876.38 


163.07 


0.177 


261 


223 
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TABLE 5.5 (continued) 



SOURCE: MORC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all sample members for whom there were 24 
months of follow-up survey data, including those with values of zero for outcomes and those who 
were assigned to JOBSTART but did not participate. 

Randon) assignment did not always take place on the first of the month. For some 
sample members, the month of random assignment may bo less than a month, beginning with the date 
of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means 
from split-file linear analysis of covariance procedures controlling for up to 29 kinds of 
difference in characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and 
Appendix Table B.4). There may be slight discrepancies in reported sums and differences of these 
adjusted means because of rounding. 

*The column labeled "p" is the statistical significance level of the difference 
between experimental and control group outcomes: that is, p is the probability that average 
outcomes are different only because of random error. A two-tailed t-test was applied to each 
difference between average experimental and control group outcomes. Statistical significance 
levels are indicated as *** = 1 percent; ** = 5 percent; = 10 percent. 
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TABLE 5.6 



IMPACTS ON YEAR-TWO EARNINGS, BY SELECTED CHARACTERISTICS 
AT THE TIME OF RANDOM ASSIGNMENT 



Characteristic and Subgroups 


Sample 
Size 


Total Earnings 
Experimental 


, Months 13-24 
Controls 


Subgroup 
Impact 




Subgroup 
Impact 
Difference^ 




laenoer 












$825.86* 


0.051 


Women 


968 


$3,014.53 


$2,827.69 


$186.84 


0.521 






Men 


871 


4,861.13 


5,500.15 


-639.02** 


0.037 


«... 




Ethnicity 














0.029 


White, non-Hispanic 


155 


5,701.77 


S, 020. 92 


680.85 


0.348 






Black, non-Hispanic 


840 


3,366.72 


3,788.33 


-421.61 


0.177 


_ _ _ 


M 


Hispanic 


783 


4,181.72 


4,104.31 


77.42 


o.eii 


_ _ _ 


W M 


Other 


61 


2,735.26 


5,873.74 


-3,138.48*** 


0.007 


_ _ _ 


M 


Ethnicity, by gender 















0.008 


Women 
















wniiG, non-nispanic 


OA 


A CO C TO 


4,447 . 13 


178.65 


0.856 






Black, non-Hispanic 


445 


2,579.67 


2,757.32 


-177.64 


0.679 






Hispanic 


413 


3,196.49 


2,670.38 


526.11 


0.236 


« « « 




Other 


26 


2,886.25 


2,134.60 


751.64 


0.677 


_ _ _ 




Men 
















White, non-Hispanic 


71 


6,889.61 


5,642.79 


1,246.82 


0.248 


_ _ ^ 




Black, non-Hispanic 


395 


4,166.06 


4,900.26 


-734.20 


0.106 


- . 


as a 


His.ianic 


370 


5,304.72 


5,730.34 


-425.62 


0.364 


— 




Other 


35 


3,100.61 


8,839.13 


-5,738.52*** 


0.000 






Parental status 














0.254 


Women living with own child{ren) 
















No 


484 


3,112.84 


2,814.14 


298.70 


0.470 






Yes 


484 


2,685.83 


2,609.55 


76.28 


0.853 






Men who have own child(ren) 
















No 


765 


4,928.60 


5,527.54 


-598.93* 


0.067 






Yes 


106 


5,435.80 


6,369.78 


-933.98 


0.293 
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TABLE 5.6 (continued) 







Total Eaminos 


. Months 13-24 






Subgroup 
Impact 
Difference^ 




Characteristic and Subgroups 


Sample 
Size 


Experimental 


Controls 


Subgroup 
Impact 






Employed within past year 
No 
Yes 


870 
969 


$3,305.70 
4,422.13 


$3,633.85 
4,518.03 


-$328.16 
-93.90 


0.285 
0.747 


-$234.26 


0.581 


Prior employment, by gender 
Women employed within past year 
No 
Yes 

Men employed within past year 
No 
Yes 


547 
421 

323 
548 


2,590.02 
3,378.85 

3,900.16 
5,582.91 


2,448.71 
3,119.44 

5,034.41 
5,924.14 


141.31 
259.41 

-1,134.25** 
-341.22 


0.715 
0.559 

0.025 
0.377 


— 


0.146 
— 


Left school in grade 11 or 12 
No 
Yes 


1,078 
761 


3,515.58 
4,428.40 


3,800.70 
4,518.67 


-285.12 
-90.27 


0.301 
0.784 


-194.85 


0.651 


Received occupational training 
within past year 

No 

Yes 


1,529 
310 


3,900.76 
3,861.27 


4,010.83 
4,504.30 


-110.08 
-643.03 


0.634 
0.210 


532.95 


0.343 


Age 
16-19 
20 or 21 


1,359 
480 


3,841.64 
4,042.95 


4,092.42 
4,117.76 


-250.78 
-74.82 


0.308 
0.857 


-175.96 


0.716 


Age, by gender 

Homen 
16-19 
20 or 21 

Men 
16-19 


710 
258 

649 


3,095.58 
2,630.77 

4,649.11 


2,801.65 
2,786.01 

5,506.21 


293.94 
-155.24 

-867.10** 


0.388 
0.782 

0.016 




0.134 



20 or 21 222 5,572.18 5,610.03 -37.85 0.951 | 
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TADLE 5.6 (continued) 







Total Earnings 


. Months 13-24 






Subgroup 

Impact 
Di f f erence^ 

W III W 1 W f 1 w w 




Characteristic and Subgroups 


Sample 
Size 


Experimental s 


Control s 


Subgroup 
Impact 


pa 

r 




Marital status 
Ever married 
Never married 


174 
1.665 


$4,992.85 
3,779.74 


$4,771.79 
4,029.01 


$221.05 
-249.27 


0.747 
0.261 


$470.33 


0.515 


Living In own household or 
with boy/girlfriend 

No 

Yes 


1,500 
339 


3,773.51 
4,441.04 


3.909.09 
4,907.89 


-135.58 
-466.85 


0.561 
0.342 


331.27 


0.543 


Own AFOC case or receiving 
General Assistance 

No 

Yes 


1.344 
495 


3,989.80 
3,630.38 


4.150.59 
3,956.47 


-160.80 
-326.09 


0.515 
0.422 


165.29 


0.728 


Own AFOC case 
No 
Yes 


1,446 
393 


3,951.62 
3,681.78 


4.145.36 
3.926.62 


-193.74 
-244.84 


0.416 
0.592 


51.10 


0.921 


Receiving Food Stamps 
No 
Yes 


1,143 
696 


3,979.86 
3,754.75 


3.984.51 
4,287.77 


-4.65 
-533.01 


0.986 
0.120 


528.37 


0.225 


Arrested since age 16 
No 
Yes 


1,567 
272 


3,928.15 
3.692.96 


4,180.31 
3,625.63 


-252.15 
67.33 


0.271 
0.903 


-319.48 


0.595 
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TABLE 5.6 (continued) 



Characteristic and Subgroups 


Sample 
Size 


Total Earninqs 
Experimental s 


. Months 13-24 
Control s 


Subgroup 
Impact 




Subgroup 
Impact 
Difference^ 


P*" 


Lived with both parents at age 14 












$695.93 


0.116 


No 


It 198 


$3,773.37 


$3,735.32 


$38.05 


0.884 


--- 


— 


Yes 


641 


4,111.52 


4,769.40 


-657.88* 


0.065 






Reason for leaving regular high school 














0.456 


School -related 


886 


4,017.62 


3,917.67 


99.95 


0.742 






Job-related 


182 


4,991.49 


5,512.52 


-521.03 


0.443 






Other 


771 


3,512.62 


3,916.94 


-404.32 


0.212 







SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 



NOTES: Calculations for this table used data for all sample members for whom there were 24 months of follow-up survey 
data, including those with values of zero for outcomes and those who were assigned to JOBSTART but did not participate. 

Average experimental and control group outcomes reported here are adjusted means from two-way analysis of 
covariance procedures controlling for up to 31 kinds of difference in characteristics, other than the characteristic used to 
define subgroups, before random assignment. The two categories used as factors were research assignment and, one at a time, 
the baseline characteristics indicated (see Ostle, 1975, p. 454). There may be slight discrepancies in reported sums and 
differences of these adjusted means because of rounding. 

'^A two-tailed t-test was applied to each wi thin-subgroup impact and also to each difference between subgroup 
impacts. For each characteristic with more than two subgroups, an F-test was applied to the interaction between that 
characteristic and experimental or control status. The columns labeled "p" are the statistical significance levels of each 
impact and each difference in impacts or F-statistic: that is, p is the probability that sample estimates are non-zero only 
because of random error. Statistical significance levels are indicated as *** « 1 percent; « 5 percent; * « 10 percent. 

For each characteristic that has only two subgroups, the subgroup impact difference is the impact within the 
first subgroup, less the impact within the second subgroup. 



(earnings) and examine how much each group was working. ^ Such an analysis shows that 
even though the proportion of experimentals working during the follow-up period drew even 
with that of controls, experimentals continued to lag behind controls in the actual time spent 
working (hours per week and weeks per month). Also, hourly wage information will be 
presented for those experimentals and controls who were working, as a way to provide some 
information on the types of jobs they found. 

A. Impacts og the Hours and Weeks of Employment 

As reported above, JOBSTART had a slightly positive impact on total two-year earnings 
for women living with their own children, a small negative impact for other women, and a 
clearly negative impact for men. This pattern carries through in hours and weeks worked, as 
shown in Table 5.7. On both measures, male experimentals worked less than did male controls 
in each year, although the difference was larger and statistically significant in the first year. 
For women living with their own children, experimentals were only slightly below controls (ihe 
differences were not statistically significant) in average hours worked per week and average 
weeks worked per month during the first year of follow-up. In the second year, experimentals 
moved slightly above controls on both measures, although the differences were again not 
statistically significant. Tlius, for the entire follow-up period, experimentals were virtually the 
same as controls on both measures. For other women, negative impacts in the first year were 
larger than small positive impacts in the second, leaving experimentals in this group slightly 
behind on both measures for the entire two-year follow-up period. 

Table 5.7 tells another interesting gender-related story. Control group men spent much 
more time working than did their control group counterparts among women - especially young 
mothers - so it is no wonder that male experimentals sacrificed more employment and 
earnings to take part in JOBSTART than did women. During the first year of follow-up, male 
controls worked an average of more than a week more per month than did controls among 
women living with their own children, and about half a week more than did other women in 
the control group. They also averaged about nine hours per week more than did women living 
with their own children, and five hours more than other women. These gender differences in 



^^This effort to understand the pattern of impacts is also the topic of Chapter 6, which discusses results 
by site. 

-151- 

ER?C 2 . 



TABLE 5.7 



IMPACTS ON WEEKS EMPLOYED PER MONTH AND 
HOURS WORKED PER WEEK THROUGH MONTH 24. 
BY GENDER AND PARENTAL STATUS 



Subgroup, Outcome, 
and Fol low-Up Period 



Experimental s Controls 



Difference 



Hen 



Average number of weeks 
employed per month during 

Months 1-24 

Months 1-12 

Months 13-24 

Average number of hours 
worked per week during 

Months 1-24 

Months 1-12 

Months 13-24 

Sample size 

Uomen living with own child(ren) 

Average number of weeks 
employed per month during 

Months 1-24 

Months 1-12 

Months 13-24 

Average number of hours 
worked per week during 

Months 1-24 

Months 1-12 

Months 13-24 

Sample size 

Uomen not living with own child(ren) , 
including those who did not have any 

Average number of weeks 
employed per month during 

Months 1-24 

Months 1-12 

Months 13-24 

Average number of hours 
worked per week during 

Months 1-24 

Months 1-12 

Months 13-24 

Sample size 



1.74 
1.32 
2.17 



15.02 
10.74 
19.30 

438 



0.78 
0.61 
0.96 



6.08 
4.56 
7.60 

250 



1.28 
0.97 
1.59 



10.20 
7.12 
13.27 

261 



1.98 
1.71 
2.26 



17.36 
14.37 
20.34 

433 



0.78 
0.66 
0.90 



6.32 
5.15 
7.50 

234 



1.35 
1.20 
1.50 



11.00 
9.40 
12.59 

223 



-0.24*** 
-0.40*** 
-0.09 



■2.34*** 
-3.64*** 
■1.04 



0.00 
■0.05 
0.06 



-0.24 
-0.59 
0.10 



■0.07 
■0.23* 
0.09 



■0.80 
-2.28** 
0.68 



0.006 
0.000 
0.432 



0.005 
0.000 
0.329 



0.971 
0.577 
0.621 



0.764 
0.483 
0.928 



0.543 
0.055 
0.534 



0.401 
0.017 
0.610 
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TABLE 5.7 (continued) 



SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all sample members for whom there were 24 
months of follow-up survey data, including those with values of zero for outcomes and those who 
were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some 
sample members, the month of random assignment may be less than a month, beginning with the date 
of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means 
from split-file linear analysis of covariance procedures controlling for up to 29 kinds of 
difference in characteristics before random assignment (see Ostle, 1975. p. 461; Cave 1987' and 
Appendix Tablt B.4). There may be slight discrepancies ^n reported sums and differences of 'these 
adjusted means because of rounding. 

«The column labeled "p" is the statistical significance level of the difference 
between experimental and control group outcomes: that is. p is the probability that average 
outcomes are different only because of random error. A two-tailed t-test was applied to each 
difference between average experimental and control group outcomes. Statistical significance 
levels are indicated as *** = i percent; ** = 5 percent; * = 10 percent. 
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controls' hours and weeks worked were much greater than the experimental-control differences 
in hours within each subgroup. 

B. Differences in Wage Rates 

Earnings are the product of time spent working (discussed above) and wages earned per 
hour. Departing from the analysis so far, the discussion in this section excludes some 
experimentals and controls (those who did not work during the periods indicated) because 
wage rates were not observed for nonworkers. Thus, the experimental-control differences 
disc -issed here are not impacts, but they do help us understand the experience underlying the 
impact findings. 

Overall, both experimentals and controls showed a gradual growth over time, although 
the trends are not always clear. Wages for working experimentals began at about $4.49 per 
hour and grew to about $5.14 per hour; for working controls, wages began at approximately 
$4.43 per hour and grew to about $5.26 per hour. Although only a 34-cent advantage for 
working controls over working experimentals in month 10 was statistically significant, working 
controls earned more per hour than working experimentals in most months. 

Splitting the sample by gender reveals that positive wage differences for working women 
almost counterbalance negative wage differences for working men each month. For example, 
in month 24, among working men, experimentals earned $5.03 and controls earned $5.39 per 
hour, for a difference of minus 36 cents, while among women, experimental earned $5.32 
and controls earned $5.01, for a difference of plus 32 cents. Among working women, 
experimentals overtook controls in hourly wages at about the same time they overtook them 
in employment and earnings. Among working men, experimentals continue to lag behind 
controls on this measure just as they do in earnings. 

V. .TOBSTARTs Impacts on Positive Activity 

A second step in understanding the short-term employment and earnings impact findings 
is to examine whether access to JOBSTART led more experimentals than controls to spend 



^^Ls is because of the likelihood of selection bias in the "choice" of those experimentals and controls 
who worked. While random assignment aeated experimental and control groups with similar characteristics 
at random assignment - except that experimentals had access to JOBSTART - it is not likely that working 
experimentals and working controls were similar except for access to JOBSTART. 
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their time in "positive activity," defined as either work or further education and training. 
Experimentals might have been working less than controls because experimentaLs were 
continuing to invest time in enhancing their skills. 

This does seem to have been the case for much of the follow-up period. Figures 5.2 
through 5.4 present impacts on positive activity for men, women living with their own children, 
and other women. For men, experimentals exceeded controls by a statistically significant 
amount for the firet 10 months of follow-up and months 15 and 16. For women living with 
their own children, experimentals exceeded controls for virtually all the first 18 months. For 
other women, experimentals exceeded controls for much of the first 10 months and then 
sporadically thereafter. 

The basic conclusion is that one importap' reason for lower earnings for experimentals 
and controls was sustained greater investment in education and training by the JOBSTART 
youths. This could signal an improved earnings situation once longer follow-up is available. 
However, during the 24 months of follow-up, no strong turnaround occurred. 

VI. .TOBSTARTs Impacts on Other Outcomes 

If JOBSTART is successful, other aspects of the JOBSTART youths' lives may also 
change. Key outcomes of interest include receipt of public benefits, criminal activity, and 
marital and parental status. Even at this early stage of follow-up, when the employment 
impact 3tory has not yet been told, it is useful to summarize briefiy early findings on these 
other aspects of the young people's lives. 

During the first 24 months of follow-up, JOBSTART had virtually no <=ipnificant pattern 
of impacts on the receipt of a long list of public benefits, including AFDC, Food Stamps, and 
unemployment insurance benefits. This general conclusion is true for the full sample and both 
women and men. The two exceptions are: (1) In some months, experimental men were more 
likdy to receive General Assistance than were control men (although the payments were never 
higher by a statistically significant amount); and (2) in some months,, experimental women were 



^or example, in the final quarter of follow-up, when employment rates and rates of productive 
activity were about equal, the average earnings of controls remained slightly (S61) above those of 
experimentals for the full impaa sample. 

-155- 

9 1 '.J 



FIGURE 5.2 

MONTHLY RATES OF POSITIVE ACTIVITY 
FOR MEN. BY RESEARCH STATUS 
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FIGURE 5.3 

MONTHLY RATES OF POSITIVE ACTIVITY 
FOR WOMEN LIVING WITH THEIR OWN CHILD(REN). 
BY RESEARCH STATUS 
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FIGURE 5.4 



MONTHLY RATES OF POSITIVE ACTIVITY 
FOR WOMEN NOT UVING WITH 
THEIR OWN CHILD(REN), BY RESEARCH STATUS 
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♦Includes women who did not have children. 



SOURCE FOR FIGURES 5.2-5.4: Appendix Table F.6. 

NOTES FOR FIGURES 5.2-5.4: Calculations for these figures used data for all sample members 
for whom there were 24 months of follow-up survey data, Including those wit.i values of zero for oi^comes 
and those who were assigned to JOBSTART but did not participate. 

■Poslttve activity" Includes JOBSTART and non-JOBSTART education, occupational 
skills training, and related activities, as well as employment. 

Random assignment did not always take place on the first of the month. For some sample members, 
the month of random assignment may be less than a month, beginning with the dato of random 
assignment and ending on the last day of the month. 
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slightly less likely to receive Supplemental Security Income (SSI) and received less in payments 
than did control women. 

By this early time in post-program follow-up, JOBSTART has not had any statistically 
significant impact on criminal behavior, as evidenced by arrests during months one through 
24 of follow-up (Table 5.8). For men and mothers, experimentals had a slightly lower rate 
of arrest than did controls, while for other women, experimentals were slightly above controls. 
However, none of the differences were large.^^ Impacts were also favorable for many other 
subgroups, but virtually none was large enough for statistical significance.^^ 

This picture differs from that produced by the previous study of the residential Job 
Corps program, but there is a clear reason to expect much less of an early impact on criminal 
behavior for JOBSTART. In the residential Job Corps program, young people moved from 
their community to a special center, which provided the education and training services. Often, 
these centers are in isolated areas or in communities without large gang populations or heavy 
involvement of youths in the drug trade. Much of the Job Corps' impact on criminal behavior 
came during the in-program period because of this "isolation" effect. In JOBSTART, no such 
change occurred in the young people's lives: They continued to live in their own 
neighborhoods and spent time outside the program with their existing circle of friends. 

A final dimension of program effects to consider is childbearing. According to Table 
5.9, among those women who were custodial parents at random assignment, assignment to the 
JOBSTART experimental group was associated with a statistically significant increase in 
childbirth by month 24; the increase was 7.3 percentage points. Among other women, the vast 
majority of whom were childless at random assignment, JOBSTART lowered the rate of 
childbirth by 2.9 percentage points, although this difference was not statistically significant. 
These findings are important because they show the extent to which childbearing may be 
inhibiting greater labor market impacts among JOBSTART women. 



Among men, 30.2 percent of experimentals and 33.4 percent of controls were arrested during the 
P^"*^- Among women, 7.6 percent of experimentals and 8.0 percent of controls were arrested, 
^e only exceptions were the impact on arrests for white, non-Hispauic youths (especially men) and 
on youths reading below the sixth-grade level at entry into he program. 
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TABLE 5.8 



IMPACTS ON CRIMINAL ARRESTS THROUGH MONTH 24. 
BY GENDER AND PARENTAL STATUS 



Subgroup. Outcome, 
and Fol low-Up Period 



Experimental s Control s 



Difference 



Men 

Ever arrested, months 1-24 {%) 

Sampl e si ze 

Uomen living with own child(ren) 

Ever arrested, months 1-24 {%) 

Sample size 

Homen not living with own chilcl(ren), 
including those who did not have any 

Ever arrested, months 1-24 {%) 

Sample size 



30.2 
438 

6.8 

250 



33.4 
433 

9.4 

234 



-3.2 



0.290 



-2.6 



0.306 



8.0 
261 



6.8 

223 



1.1 



0.645 



SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all sample members for whom there were 24 
months of follow-up survey data, including those with values of zero for outcomes and chose who 
were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some 
sample members, the month of random assignment may be less than a month, beginning with the date 
of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means 
from split-file linear analysis of covariance procedures controlling for up to 29 kinds of 
difference in characteristics before random assignment (see Ostle, 1975. p. 461; Cave. 1987; and 
Appendix Table B.4). There may be slight discrepancies in reported sums and differences of these 
adjusted means because of rounding. 

*The column labeled "p" is the statistical significance level of the difference 
between experimental and control group outcomes: that is, p is the probability that average 
outcomes are different only because of random error. A two-tailed t-test was applied to each 
difference between average experimental and control group outcomes. Statistical significance 
levels are indicated as *** « 1 percent; ** = 5 percent; * = 10 percent. 
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TABLE 5.9 



IMPACTS ON PREGNANCY AND CHILDBIRTH THROUGH MONTH 24. 
BY PARENTAL STATUS AT THE TIME OF RANDOM ASSIGNMENT 



Subgroup, Outcome, 
and Follow-Up Period 



Experimental s Control s 



Di fference 



Uomen living with own child(ren) 

Ever pregnant, months 1-24 (%) 
Ever gave birth, months 1-24 (%) 

Sample size 

yomen not living with own chilcl(r€n), 
including those who did not have any 



Ever pregnant, months 1-24 (%) 
Ever gave birth, months 1-24 



Ever g 
Sample size 



) 



59.1 
32.7 

250 



45*6 
18*5 

261 



53.1 
25.4 

234 



47.5 
21.4 

223 



6.0 
7.3^ 



■1.9 
-2.9 



0.191 
0.087 



0.678 
0.443 



SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all sample members for whom there were 24 
months of follow-up survey data, including those with values of zero for outcomes and those who 
were assigned to JOBSTART but did not participate* 

Random assignment did not always take place on the first of the month. For some 
sample members, the month of random assignment may be less than a month, beginning with the date 
of random assignment and ending on th» last day of the month. 

Average experimental and control group outcomes reported here are adjusted means 
from split-file linear analysis of covariance procedures controlling for up to 29 kinds of 
difference in characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and 
Appendix Table B.4). There may be slight discrepancies in reported sums and differences of these 
adjusted means because of rounding. 

*The column labeled "p" is the statistical significance level of the difference 
between experimental and control group outcomes: that is, pis the probability that average 
"^comes are different only because of random error. A two-tailed t-test was applied to each 
diiference between average experimental and control group outcomes. Statistical significance 
levels are indicated as *** » 1 percent; ** « 5 percent; * = 10 percent. 
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CHAPTER 6 



SITE VARIATION IN IMPACTS 

Chapters 4 and 5 provided estimates of JOBSTARTs impacts on educational attainment, 
employment, and earnings for the full sample and for key demographic subgroups. This 
chapter addresses another important issue: How were impacts affected by program 
characteristics? The most direct way to explore this issue within the JOBSTART 
Demonstration is to examine differences in individual site impacts, since there were some 
programmatic differences among the 13 sites and the programs operated in a variety of settings. 
For reasons soon to be discussed, this analysis of impacts across sites, of necessity, is less 
certain than that reported in the previous two chapters. 

While the limited but still relatively large number of sites in JOBSTART is clearly an 
advantage in this effort, it does not assure succesi^^ in "teasing out" how program characteristics 
affected impacts. As with virtually all multi-site demonstrations, the JOBSTART Demonstration 
was not designed to address, with the rigor of an experiment, the question of how differences 
in program structure influence impacts. The major goal of the demonstration was to ascertain 
whether the model, as implemented in a diverse sample of sites, would lead to improved 
employment and earnings. For this purpose, youths in each site were randomly assigned to the 
experimental and control groups. 

To answer, with similar rigor, the question of how program characteristics affect impacts, 
youths in individual sites would have had to be randomly assigned to one of the several types 
of programs of interest (for example, those emphasizing education versus those emphasizing 
training), and individual sites would have had to offer more than one type of program. This 
kind of study - known as a "differential impact" research design - would be needed to 
separate out cleariy the influence of program type from other site variations such as local labor 
market conditions and differences in the kinds of youths drawn to the various sites. ^ This 
approach was not followed in light of the main goal of the study: to use the combined cross- 
site sample to answer impact questions about the full sample and subgroups. 



^For example, as discussed in Chapter 2, sites in the JOBSTART Demonstration did vary in the 
proportions of their youths who had prior work experience and welfare receipt, and in the proportion with 
children and from the black and Hispanic ethnic groups. 
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Nevertheless, the JOBSTART Demonstration did provide an opportunity to learn more 
about the link between program characteristics and impacts. For example, if the respective 
categories of site programs had shown clear differences in impacts, certain lessons might have 
been drawn (including the lack of influence of other factors). As the findings in this chapter 
show, however, no clear pattern emerged: That is, no single category of sites did appreciably 
better than another in terms of two-year impacts. Instead, both positive and negative impacts 
appeared within each category, implying that JOBSTART can produce positive (or negative) 
impacts in a variety of settings and with a variety of program structuru. No single approach 
either assures positive impacts or is necessary for them. 

Before turning to the analysis per se, some additional preliminaries may be useful. Much 
of the previous analysis in this report divided sites into three groups - concurrent, 
sequential/in-house, and sequential/brokered - because these categories helped explain 
differences in program implementation and participation among the sites.^ This chapter 
preserves these site groupings, since the implementation research led to a hypothesis that these 
differences in program structure could influence impacts. But it is important to keep in mind 
that these categories are not a neat division of the sites into homogeneous groups. One key 
difference within the concurrent category was discussed in Chapter 1 and elsewhere in the 
report: Two sites (CET/San Jose and Chicago Commons) integrated education into the training 
curriculum to a much greater extent than did the other concurrent sites. Other characteristics 
that varied within categories included the extent to which applicants were initially screened, the 
quality of sites' implementation of the four JOBSTART components, and labor market 
conditions. Thus, disentangling the independent effect of any single program feature on 
impacts across sites can be very difficult. 

One final point should be made: The impacts on employment and earnings at the 
individual site level (even when noticeably different from zero) were usually not statistically 
significant.^ There are three reasom, for this: (1) Most of the reported employment and 
earnings impacts were not large; (2) a site's program did not affect the behavior of all youths 
in the same way, and controls' earnings also varied (in technical terms, this suggests a large 



^For example, the sequential/brokered sites all had difficulty getting young people to make the 
transition to training, and the concurrent sites all tended to emphasize training (as opposed to education) 
more than the other sites. 

^This was not true for site-level impacts on educational attainment, which in many sites were larg^ 
and statistically significant. 
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variance in the outcomes); and (3) the sample size in each of the sites was relatively small. 
Furthermore, the differences in the individual site impacts for employment and earnings were 
also not statistically significant, suggesting again that lessons on program structure must be 
tentative."* 

To summarize the implications of these fairly technical issues: Only if there was a clear 
pattern of impacts among types of sites should lessons be drawn from this analysis about how 
one particular program feature affected impacts. Furthermore, in general, conclusions should 
not be drawn from the impacts at any single site. 

Despite these difficulties of analysis and interpretation, this chapter does try to draw 
some conclusions about the link between program features and impacts. It moves into thi;> 
type of analysis because of the strong interest in identifying more effective ways to structure 
education and training programs, an interest intensified by the recent publication of the first 
impact report from the Minority Female Single Parent (MFSP) Demonstration, sponsored by 
the Rockefeller Foundation.^ That study analyzed program impacts in four sites offering very 
different types of education and training for low-income, minority mothers who averaged 28 
years of age. One site, the Center for Employment Training (CET) in the San Francisco Bay 
area, offered an integrated program of training and education, with little initial screening of 
applicants and intensive job placement efforts. This site showed strong employment and 
earnings impacts in the fourth quarter of follow-up, while in the remaining three sites - each 
with an emphasis on basic skills instruction, which was to be followed by various types of 
occupational training — experimentals and controls had approximately equal employment and 
earnings. Some have attributed this difference in impacts between CET and the other sites to 
specific features of its program, especially its integration of education and training. CET/San 
Jose was a part of both the JOBSTART and MFSP demonstrations, although in JOBSTART, 
the CET/San Jose sample was made up primarily of men and women without children, since 
most young mothers were part of the MFSP Demonstration. 

This chapter proceeds with a framework for analyzing site-level impacts. It then discusses 
site-level impacts for the full impact sample and for the three key subgroups of men, women 



"^Statistical significance in this context is a test of whether the extent of variation in impact<^ across 
sites was so systematic that it was unlikely to have arisen by chance. 
^See Gordon and Burghardt, 1990. 
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living with their own children, and other women. It closes with a summary of the findings and 
directions for future research. 

L A Framework for Analvzine Site Variation in Impacts 

This section discusses how JOBSTART sites varied in ways that could influence program 
impacts. It then presents two alternative types of site impact estimates used in this chapter. 

A. Factors Lading to Site Variation in Impacts 

The differences in program structure among the sites, discussed earlier in this report, 
were among the most obvious possible influences on site impacts. However, these program 
factors were entangled and confounded across JOBSTART sites with several other sources of 
variation in impacts including the target group recruited at various sites and their relative 
interest in education per se as opposed to training, the extent of screening by site operators, 
the point at which random assignment was placed for the evaluation, the availability of local 
alternative services for members of the control group, and environmental factors such as the 
wage structui e and tightness of the labor market. 

1* Program structure* As detailed in Chapter 3, dimensions of program structure 
that are of special interest include concurrent versus sequential education and training classes, 
the extent of integration of education and training, months of program activities offered and 
delivered, brokering of services among multiple vendors versus in-house provision of all services, 
the relative emphasis on education as opposed to occupational skills training, the strength of 
job placement efforts, and the intensity of support services. Table 6.1 combines selected 
fmdings from earlier tables to highlight key aspects of the sites. Sites are grouped under the 
headings "concurrent," "sequentiaHn-house," and "sequential/brokered." 

"Length of stay" is an important dimension of site variation when examining follow-up in 
a short period such as two years. If programs were able to provide equal hours of instruction, 
in sites in which experimental participated for shorter periods of time, there will have been 
more post-program time during which experimental employment outcomes could have overtaken 
control outcomes at the two-year point. CET/San Jose and Chicago Commons are notable on 
this score, with average lengths of stay less than two-thirds of the full sample average, even 
though total hours were higher than average at Chicago and 88 percent of the average at 
CET/San Jose. The "average hours per month" column is a measure of the extent to which 
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TABLE 6.1 

SUMMARY OF JOBSTART IMPLEMENTATION, BY SITE 





Total 


Length 


Average 


Hours 


of 


L?ve1 of 


Rating of 


Overall 


Site 


of Stay 


Hours 






Initial 


Job 


Support 


Rating of 


Hours 


(Months) 


per Month 


Education 


Training 


Screening 


Placement 


Services 


Implementation 


Concurrent 


405 


6.4 


60 


104 


291 










Atlanta Job Corps 


296 


5.5 


50 


95 


150 


High 


Low 


High 


Low 


CET/San Jose^ 


366 


4.4 


74 


29° 


337 


Low 


High 


Med i um 


High 


Chicago Commons* 


445 


4.5 


83 


72b 


373 


High 


High 


Medium 


High 


Connelley (Pittsburgh) 


579 


10.1 


54 


105 


473 


Medium 


Medium 


High 


High 


East LA Skills Center 


345 


5.4 


55 


73 


272 


Medium 


Low 


Medi um 


Medium 


EGOS (Denver) 


274 


7.1 


33 


126 


148 


Low 


Low 


Medium 


Low 


Phoenix Job Corps 


441 


6.5 


60 


163 


218 


High 


High 


High 


High 


SER/Corpus Christi 


418 


5.2 


76 


124 


294 


Medium 


Low 


High 


Medium 


Sequential 1 in-house 


536 


7.1 


66 


152 


287 










El Centro (Dallas) 


401 


5.8 


60 


142 


179 


Medium 


Medium 


Medium 


High 


LA Job Corps 


631 


8.1 


71 


159 


362 


High 


Medium 


High 


High 


Sequen tiaU broke red 


316 


7.8 


37 


178 


78 










Allentown (Buffalo) 


440 


10.7 


39 


244 


113 


Medium 


Low 


High 


Medium 


BSA (NYC) 


280 


5.5 


45 


145 


66 


Low 


Low 


Medium 


Low 


CREC (Ha» tford) 


166 


6.3 


23 


119 


36 


Low 


Low 


Medium 


Low 


All sites 


415 


6.8 


57 


128 


249 











SOURCES: MORC calculations from JOBSTART enrollment fonn» MIS» and survey data (participation figures); MDRC operations staff 
(implementation ratings). 



NOTES: *In this site, education and training were more integrated than in other sites» and staff strongly emphasized training 
over passing the GED examinatioii. 

'^In this site, some education hours are included in the training component hours. 



education, training, and other hours were concentrated or spread out over time. In general, 
the shorter the length of stay, the higher were the hours per month. CET/San Jose and 
Chicago Commons, the two sites with the shortest length of stay, had high average hours per 
month because they squeezed just about as much program activity as the average site had into 
shorter periods of time. 

CET/San Jose and Chicago Commons had two things in common that help explain their 
relatively short length of stay. First, the programs were concurrent, offering education and 
training at the same time rather than one after the other. Second, there was less emphasis on 
education leading to a GED than on skills training in these integrated or partly integrated 
programs. Thus, youths in these programs typically litayed fewer months than those in 
sequential programs and in other concurrent programs that emphasized education more. For 
these reasons, one might hypothesize that second-year earnings impacts in these two sites might 
be greater than those in sites that provided the same level of services over a longer period of 
time. Opportunity costs (forgone employment and earnings) were smaller, and experimental 
in the two sites had more time to catch up with and overtake controls in employment rates and 
earnings by month 24. One might also hypothesize that sites providing more services over a 
longer penod could have greater longer-term impacts. This issue will be clarified when the 
four-year follow-up results are known. 

The "hours of education" column shows much variation, but there was a tendency for 
the sequential - and especially sequential/brokered ~ sites to emphasize education more than 
the concurrent sites. Four of the eight concurrent sites averaged more than 50 hours less than 
the average for all sites, and only one of them exceeded the all-sites average. It stands to 
reason that concurrent sites would have delivered fewer hours of education than sequential 
sites because concurrent programs had to squeeze more activities into the school day. 
Vrriation in the proportion of youths getting any education was not a major factor in site 
variation in hours: Table 3.9 showed that, except at CET/San Jose (which did not offer 
separate education classes), this proportion varied in a narrow range around the full sample 
average of 86 percent. 

The "hours of training" column shows more site variation than the previous columns. As 
already indicated, the dramatic difference between the sequential/brokered sites and the others 
on this score probably reflected both the systemic and administrative problems the former 
encountered in linking education and training and differences in the interests and expectations 
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of recruits. Table 3.10 showed that only one quarter of esperimentals at sequential/brokeied 
sites got any occupational skills training at all. One surprise here is that the two 
sequential/in-house programs managed to deliver hours of training comparable to the hours in 
concurrent programs. This achievement is notable, since only half the experimental in 
sequential/m-house sites stayed long enough to receive any training, while more than 80 percent 
of experimental in concurrent sites received training. 

The "job placement" column repeats the subjective ranking given in Table 3.4 for the 
strength of job placement assistance efforts in a site. CET/San Jose, Chicago Commons, and 
the Phoenix Job Corps were considered especially strong on this score; Connelley in Pittsburgh, 
El Centro in Dallas, and the Los Angeles Job Corps were considered middling; and all the 
other sites were considered weak. Looking at this column by itself would lead one to predict 
high labor market impacts early on at CET/San Jose, Chicago Commons, and the Phoenix Job 
Corps. However, the effectiveness of various types of job placement assistance could be 
affected by labor market conditions. 

The "support services" column is taken from Table 3.4 and assesses the strength of the 
implementation of the fourth component of JOBSTART, support services. Along with the 
three Job Corps nonresidential programs, Connelley in Pittsburgh, SER/Corpus Christi, and 
Allentown in Buffalo stood out from the other sites in the strength of their efforts to deal 
with problems that stood in the way of program attendance and completion. 

To sum up this table, CET/San Jose, Chicago Commons, Connelley in Pittsburgh, El 
Centro in Dallas, and two of the three Job Corps programs seem to have implemented 
JOBSTARTs four components more fully than did the other sites. Thus, these sites appear 
to have set the stage for program impacts stronger than those in the other sites. However, the 
relatively strong treatments, short lengths of stay, and resulting intensity (hours per month) at 
CET/San Jose and Chicago Commons could have given these two sites an edge over the others 
in a short follow-up period. 

2. Recruitment While all JOBSTART sites had a common target group for the 
demonstration (disadvantaged young dropouts reading below the eighth-grade level), there 
were variations in site recruiting emphases and in participant characteristics. These site 
variations arose because different types of youths applied and because intake practices were 
not uniform. 

Some characteristics of youths vary in easily measured ways. For example, as reported 
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in Chapter 2, sites varied greatly in the proportion of the sample that was made up of youths 
who had recent work experience or who were young mothers. Factors such as these were 
measured at random assignment and can, to a considerable extent, be adjusted for statistically 
in comparing program impacts across sites. One type of impact estimate reported later in this 
chapter does include such statistical adjustments in an effort to control for this one source of 
variation among sites and to move closer to isolating variation in impacts caused by program 



structure.^ 



Other factors are much harder to observe and were, therefore, not measured at entry 
into the sample; these cannot be included in statistical adjustments. One especially important 
factor is the youths' goals in participating in the program and their resulting interest in the 
various JOBSTART components. Although the youths' goals and interests were not observed 
directly in any quantifiable way, the reputation of the site in the community probably had a 
strong influence on who applied for the program. Those sites with a history of providing basic 
skills instruction and not occupational training (BSA in New York City, Allentown in Buffalo, 
and CREC in Hartford) naturally would seem to have attracted youths who were more 
interested in GED attainment than in immediate acquisition of marketable occupational skills, 
while those sites emphasizing occupational skills training (CET/San Jose and Chicago 
Commons) seem to have attracted those who were more interested in job skills and immediate 
employment than in education. 

Available information does indirectly support this generalization. Table 3.10 showed that 
JOBSTART hours of education tended to be high and that hours of occupational skills training 
were lowest in sequentiaVbrokered sites, although the latter clearly resulted partly from the 
administrative problems those sites encountered in linking education and training. Taken 
together, these findings are consistent with the idea that recruits in the sequential/ brokered 
sites were less interested in occupational training per se than in education leading to a GED. 
In other words, these sites may have delivered fewer hours of skills training partly because 
their recruits demanded fewer hours. 

3. Screening. Some sites - such as CET/San Jose, EGOS in Denver, BSA in New 



^Although these impacts include linear statistical adjustments for these characteristics, r.ot all relevant 
characteristics were measured; there might be differential errors of measurement of characteristics; true 
relationships between impacts and characteristics may be nonlinear; or impacts of sites with very Mn^x.^l^\ 
sample characteristics might be difficult to adjust properly with any statistical model. Thus, adjusted site 
outcomes and impacts must be viewed with a special caution not applicable to estimated subgroup impdcii. 
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York City, and CREC in Hartford - did not screen applicants much before they entered the 
sample, while others (such as Chicago Commons and the Job Corps sites) carried out extensive 
screening. When a great deal of screening takes place, it is reasonable to assume that those 
randomly assigned (including those assigned to control group status) will have high levels of 
perseverance, motivation to go into a program, motivation to work, and other important 
characteristics hard to measure directly. Thus, all else being equal, in sites that screened 
heavily, control group outcomes should have constituted tougher benchmarks for experimental 
to surpass than in sites that carried out minimal screening. However, minimal screening may 
yield groups of experimentals who are harder to keep in the program and harder to place at 
completion. 

4. Point of random assignment On top of the normal steps and resulting screening 
in recruitment and intake, the research design introduced random assignment, which could 
occur at slightly different points relative to initial contact with a potential participant and actual 
start-up of services. As mentioned in Chapter 2, putting random assignment early in the 
program flow (as at CET/San Jose) tended to include in the sample individuals v/ho might drop 
out of the later steps of intake, while putting random assignment relatively late (as at 
Allentown in Buffalo) meant that only those making it to that stage of intake were in the 
research sample. Late random assignment tended to raise the proportion of the experimental 
group participating in JOBSTART, because it meant that those randomly assigned were 
interested enough in the program to persevere through the steps of intake and because it 
reduced the wait between random assignment and the start of services. It also tended to raise 
the proportion of controls who were served in alternative programs, because those randomly 
assigned to the control group were also quite motivated to receive services.^ 

5. Service availability. More programs tend to be available in large cities, such as 



Chapter 3 pointed out the severe recruitment pressures at CET/Szu Jose during intake tor 
JOBSTART. CET/San Jose may have been able to adopt this "no screeDing* no\i^ because of the 
characteristics of disadvantaged youths in its service area. Even with open ad.n/s.ions, « higher j^.-icentage 
of the CET/San Jose youths had recent work experience than did the full sauiple. 'iTiis prnbabiy occurred 
because CET/San Jose drew largely Hispanic youths, a group that, as mentioned in Ch apter i, typically ' as 
a higher labor force participation rate than do other minority school dropouts. 

^e correlation coefficient of sites' percentage of experimentals and controls participating in 
education and training is +.37. The point of random assignment thus had implications for the -rovtorcion 
of expenmentals and controls receiving services, but both groups were affected simiiarly, so there W. no 
consistent effect on the difference in the proportion of experimentals and controls ret-eiving ^ome <ype of 
education and training. 
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New York, Chicago, and Los Angeles, than in smaller cities such as Corpus Christi or Hartford. 
Since the impacts reported here are the impact of JOBSTART compared to the alternative 
services in which the controls participated (rather than the impacts of JOBSTART services 
versus no services), variation in the controls' benchmark levels of services wiU influence impacts. 
As Table 6.2 shows, the level of control services and the resulting difference between 
experimentals and controls in service receipt varied greatly by site.^ For the most part, service 
receipt differences were smaller in sequential/brokered sites than in other sites. The largest 
service differentials were found at El Centro in Dallas and SER/Corpus Christi. 

6. Wage structure and labor market tightness. In some communities, jobs tend to 
be plentiful but low-paying; in others, they tend to be scarcer but higher-paying; and in still 
others, they are both scarce and low-paying. This could be the result of long-term differences 
in the industrial base of the communities or of how they are affected by business cycles. These 
labor market conditions could affect both the control group's level of employment and earnings 
and the availability of jobs that reward the experimental group's increased educational 
attainment and occupational training. 

B. Two Alternative Types of Site Impact Estimates 

In this chapter, impacts are compared across sites to help explain how differences in 
program operation affected program impacts. M just discussed, however, sites differed in 
many ways, making it very difficult to isolate the influence of program features. The most 
straightforward approach if to calculate separate experimental-control comparisons for each 
site, in effect creating 13 separate samples (or data files) and calculating impacts for each one. 
In doing this type of site impact analysis, the only adjustment made is to take account of 
differences in the observed pre-random assignment characteristics of the experimental and 
control groups, which may occur when sample sizes are relatively small. 

It is possible to move beyond this "split-file" analysis because one type of site variation 



*This table is taken from part of Table 4.1. Differences in service receipt across sites were statistically 
significant. Alternative estimates of service receipt, which result from two-way analysis of the effect of site 
and experimental or control status, are not materially different from those in Table 6.2 (see Appendix 

Table G.l). ... . ^ ■ 

^®As discussed in Chapters 2 and 3, at CET/San Jose some early activities were not counted in the 
JOBSTART data system; thus, the artual participation rate for experimentals was somewhat higher than 
the data suggest. Nevertheless, the reported experimental-control difference in service receipt for CET/San 
Jose is higher than average. 
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TABLE 6,2 



EXPERIMENTAL-CONTROL DIFFERENCE IN SERVICE RECEIPT, BY SITE 



Ever Received Any Education 
Sample or Training, Months 1-24 
Site Size Experimental s Controls Difference p 



Concurrent 

Atlanta Job Corps 
CET/San Jose 
Chicago Commons 
Connelley (Pittsburgh) 
East LA Skills Center 
EGOS (Denver) 
Phoenix Job Corps 
SER/Corpus Christi 

Sequential I in-house 

El Centro (Dallas) 
LA Job Corps 

Sequential I brokered 

AUentown (Buffalo) 

?SA (NYC) 

CREC (Hartford) 

All sites 



SOURCE: MDRC calculations from JOBSTART enrollment form, MIS, and survey data. 

NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

"Any education or training" includes JOBSTART and non-JOBSTART education, 
occupational skills training, and related activities* 

Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted 
means from split-file linear analysis of covariance procedures controlling for 19 kinds of 
difference in characteristics before random assignment; "all sites" outcomes are from a 
linear analysis of covariance procedure for the full sample controlling for up to 31 kinds 
of difference in characteristics before random assignment (see Ostle, 1975, p, 461; Cave, 
1987; and Appendix Table B,4), There may be slight discrepancies in reported sums and 
differences of these adjusted means because of rounding, 

^The column labeled "p" is the statistical significance level of the 
difference between experimental and control group outcomes: that is, p is the probability 
that average outcomes are different only because of random error, A two-tailed t-test was 
applied to each difference between average experimental and control group outcomes. 
Statistical significance levels are indicated as *** « 1 percent; ** » 5 percent; * - 10 
percent. 



61 92,7% 47,9V 44,8*** 0,001 

152 73.1 21.0 52.0*** 0,000 

74 93.2 60.9 32.3*** 0.004 

184 99.8 45.8 54.0*** 0.000 

100 92.2 59.1 33.2*** 0.000 

183 94.2 51.6 42.6*** 0.000 

130 92.5 36.6 55.9*** 0.000 

236 98.8 34.6 64.2*** 0.000 



155 99.3 27.8 71.6*** 0.000 

218 87.1 45.5 41.6*** 0.000 



140 97.8 70.6 27.2*** 0.000 

119 88.3 49.3 39.1*** 0.000 

87 93.5 47.4 46.1*** 0.000 

1,839 92.7 44.2 48.4*** 0.000 
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- the observed characteristics of the youths in the sample - can be adjusted for statistically 
by relying on the fact that youths with particular characteristics are rarely concentrated in just 
one site. This type of adjustment takes account of the program impacts for each type of youth 
in the full sample (for example, the impacts for blacks, those with prior work experence, or 
those receiving public assistance) and adjusts individual site impacts to reflect the fact that 
some sites served many youths who (in the full sample) tended to have lower-than-average 
impacts, while other sites served more than the usual percentage of youths who had higher- 
than-average impacts. 

The two approaches answer different questions, and each has strengths and weaknesses. 
The split-file approach is most valuable as a description of what actually occurred in each site: 
That is, it presents the best estimates for the impacts of the site's program as it actually 
operated for the people recruited and randomly assigned to the experimental group. This is 
the approach taken, for example, in tlic recent report on the MFSP Demonstration; the 
findings for each of the four sites were calculated separately. Its chief drawback is that the 
differences in outcomes for experimentals and controls within each site are not fully comparable 
across sites because the characteristics of the two groups differ from site to site. If, for 
example, youths who were employed in the year prior to random assignment tended to have 
lower employment and earnings impacts, site*^ '^'\h a sample made up of youths with higher- 
than-average prior-year employment will have impacts lower than they would have been had 
those sites served a more disadvantaged mix of recruits. Furthermore, the individual site files 
may have relatively small sample sizes, limiting the precision of impact estimates. 

The alternative of adjusting for measured site differences in pre-random assignment 
characteristics is most valuable in trying to isolate the effects of location per se. Sites with 
populations different from the overall average are not penalized or rewarded relative to other 



^^For example. Chapter 5 discussed how impacts were more favorable for women living with their own 
children than for other women or for men. The adjustment would take account of the fact that the sample 
in some sites included more than the average proportion of women living with their own children (thereby 
raising observed site impacts) and some served more than the average proportion of men (thereby lowering 
observed site impacts). The adjustment was designed to remove the difference in impacts arising from 
theie differences in each site's sample. The statistical adjustment assumes a linear relationship between 
each characteristic and outcomes and is the same as the method underlying Tables 4.4, 4.7, 5.3, and 5.6, 
which presented impacts for designated subgroups controlling for differences between the subgroups other 
than the characteristic used to define them. It used linear adjustments of outcomes, with up to 19 kinds 
of differences in youths' characteristics entered as covariates. See Ostle, 1975. 

^^ee Gordon and Burghardt, 1990. 



-172- 



sites, as they are in the split-file approach. To stay with the same pxample: The estimated 
impacts for a site with a higher-than-average proportion of youths with prior employment would 
be larger using this adjustment than with the split-file approach. However, in order to make 
the adjustment for site differences in the mix of youths* pre-random assignment characteristics, 
it is necessary to assume that sites could have served - and gotten "average" impacts for - 
populations they actually did not serve. For example, sites set up to serve a particular type 
of person (for example, young men) might not be able to serve other young people. Thus, 
they might not attain the impact observed in the full sample for these other groups. Each 
way of looking at site impacts answers a particular question and has its own advantages and 
disadvantages, so both arc presented in the following section. 

For both approaches, the impacts presented are per experimental. Appendix B, Section 
III, discusses how impacts per experimental can be converted to impacts per participant if 
certain assumptions are valid. This conversion was not done here - despite the fact that 
participation rates for experimentals do vary across the sites - because participation rates for 
controls also vary, and are somewhat correlated with the experimental' rates: For example, 
sites with a high participation rate for experimentals tend to have a high rate for controls. 
Consequently, comparisons of the difference in participation between experimentals and controls 
across sites show a different pattern of variation than comparisons of the participation rate for 
either group individually: For instance, sites with a lower-than-average participation rate for 
experimentals do not necessarily have a lower-than-average difference in participation between 
experimentals and controls. Since the impact analysis presented here compares experimental 
and control outcomes and shows the impact of the increment of services received by 
experimentals above the services received by controls, adjusting site-specific impacts to account 



adjusted impacts approach uses the entire sample, lessening the problems of sample size in 
the analysis. However, it introduces a further complication: The adjustments implicitly assume that 
youths' characteristics affect outcomes in the same way in each site. More technically^ the assumption is 
that the coefficients estimated for the regression used in making adjustment for differences in 
characteristics (both between experimentals and controls and across sites) are the same acrcss all sites. 
The split-file approach estimates separate regressions for each site to adjust for differences in sample 
characteristics between experimentals and controls. Therefore, the shift from split-file impacts to those 
adjusted for site differences in pre-random assignment characteristics involves two types of changes: 
changing to the standard full sample regression coefficients and adjusting for differences across sites in 
sample characteristics. Consequently, it is often not possible to give a simple, intuitive explanation of why 
the shift from split-file to a isted impacts caused the observed change in impacts. 
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for cross-site variation in cxperimentals' participation would confuse rather than clarify 
comparisons across the sites. 

II. Full Sample ImpactSt by Site 

The outline of the site story is the same under both types of site impact approaches, 
although the adjustment does make a noticeable difference in the impacts of a few sites that 
served a group of young people that differed markedly from the sample as a whole and/or had 
smaller-than-average sample sizes. Table 6.3 presents the split-file impacts, that is, those with 
no adjustments for differences across sites in the youths served. Table 6.4 presents impacts 
with that type of adjustment. Both tables include three impacts: The left section presents 
experimental-control differences in educational attainment; the middle section, differences in 
employment rate during the second year of follow-up; and the right section, differences in 
earnings during the second year of follow-up. 

As anticipated, few of the individual site impacts were statistically significant, so lessons 
would come from patterns of impacts across sites. In both tables, in each site grouping, there 
are one or more sites with a positive impact and one or more with a negative or very small 
impact for each of the three outcomes: high school completion and GED receipt, and 
employment and earnings in the second year. As the tables show, there was variation in 
impacts within each group of sites as well as between groups of sites. For educational 
attainment, the differences in site impacts were statistically significant.^^ However, the 
differences in the individual site impacts on employment and education were not statistically 
significant, suggesting caution in drawing conclusions from any observed differences in impacts. 

High school diploma and GED receipt impacts were mainly positive (and, in four sites, 
statistically significant) but fell negative at Cu^cago Commons and the East Los Angeles Skills 



^''It is possible to adjust impacts to take account of participation in services by both experimentals 
and controls, but this involves making untestable assumptions about the impacts of the services received 
by controls. See Auspos, Cave, and Long, 1988, pp. 198-199. 

^^Table 6.4 includes a test of the statistical significance of the variation in individual site impacts. The 
"p-value" of the T-statistic" at the foot of each column in Table 6.4 is the probability that site variation 
in that impact could be due to chance alone. For educational attainment, the observed site variation in 
impacts was so great that the probability it could have .arisen by chance is virtually zero. However, for 
both employment and earnings, individual site impact estimates are less diverse, as is their precision 
(represented by a general lack of statistical significance on site impacts), so the probability that the 
observed variation could have arisen by chance is much higher: 72 percent for employment and 13 percent 
for earnings. 
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TABLE 6.3 



SELECTED IMPACTS OF JOBSTART, BY SITE 
(NOT ADJUSTED FOR SITE DIFFERENCES IN SAMPLE CHARACTERISTICS 
AT THE TIME OF RANDOM ASSIGNMENT) 



0 1 tc 


Sample 
31 ze 


Received GEO 
or High School 
Diploma by End of 
Month 24 


U1 TTerence 


E' " Employed, 
Months 13-24 


Di fference 


Total Earnings, 
Months 13-24 


Difference 


Expert - 
mentals 


Controls 


Experi - 
mentals 


Controls 


Experi - 
iitentals 


Controls 
























Atlanta Job Corps 


61 


23.0% 


8.6% 


14.3 


65.4% 


76.5% 


-11.0 


$2,547.85 


$4,853.46 


-2,305.61* 


Lci/oan uosc 


1 C9 

IDC 


34.5 


27.4 


7 . 1 


90.0 


86.3 


3.7 


7,827.81 


7.319.39 


508.42 


Chicago Conwnons 


74 




18.7 


-13.1 


77.0 


68.9 


8.1 


4,236.06 


4,158.49 


77.56 


uonnciicjr \r1wLSDur9ny 




57.1 


15.9 




68.5 


58.5 


10.0 


1,637.28 


2,093.65 


-456.38 


East LA SMI Is Center 


100 


2 . 1 


n A 

8.0 


-5.9 


75.4 


70.5 


4.8 


4,619.64 


5,513.23 


-893.59 


EGOS (Denver) 


183 


22.2 


14.8 


7.4 


63.8 


65.1 


-1.3 


2,823.37 


3,553.35 


-729.99 


Phoenix Job Corps 


130 


25.2 


11.3 


13.9* 


71.4 


81.2 


-9.8 


3,241.16 


4,623.10 


-1,381.95* 




L JU 


53.2 


15.1 


Jo . l""" 


77.4 


78.5 


-1.2 


3,684.59 


3,575.35 


109.23 
























El Centro (Dallas) 


155 


57.5 


14.1 


43.4*** 


77.5 


55.2 


12.3 


3,926.87 


3,363.75 


563.12 


LA Job Corps 


218 


19.8 


14.8 


5.1 


64.3 


62.2 


Z.l 


3,960.76 


4,418.62 


-457.86 


Sequen tiall brokered 






















Allcntown (Buffalo) 


140 


33.7 


21.3 


12.4 


63.1 


64.1 


-0.9 


2,843.72 


2.370.42 


473.31 


BSA (NYC) 


119 


25.7 


32.0 


-6.3 


67.1 


74.8 


-7.7 


4,687.46 


6,074.06 


-1,386.61 


CREC (Hartford) 


87 


21.8 


12.4 


9.4 


69.2 


64.0 


5.2 


4,333.44 


4,627.33 


-293.89 


All sites 


1,839 


33. Z 


16.4 


16.7*** 


72.0 


69.5 


2.5 


3,894.08 


4,098.81 


-204.73 



SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 



NOTES: Calculations for this table used data for all sample members for whom there were 24 months of follow-up survey data, including 
those with values of zero for outcomes and those who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample members, the month of random assign?nent 
may be less than a months beginning with the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means from split-file linear analysis of covariance 
procedures controlling for 19 kinds of difference in characteristics before random assignment; •'all sites" outcomes are from a linear analysis 
of covariance procedure for the full sample controlling for up to 31 kinds of difference in characteristics before random assignment (see 
Ostle, 1975, p. 461; Cave, 1987; and Appendix Table B.4). There may be slight discrepancies in reported sums and differences of these 
adjusted means because of rounding. 

A two-tailed t-test was applied to each difference between average experimental and control group outcomes. Statistical 
significance levels are indicated as *** ■ 1 percent; « 5 percent; * • 10 percent. 



TABLE 6.4 



SELECTED IMPACTS OF JOBSTART, BY SITE 
(ADJUSTED FOR SITE DIFFERENCES IN SAMPLE CHARACTERISTICS 
AT THE TIME OF RANDOM ASSIGNMENT) 



Site or Statistic 


Sample 
Size 


Received GEO 
or High School 
Diploma by End of 
Month 24 




Ever Employed, 
Months 13-24 


Impact 


Total Earnings, 
Months 13-24 


Impact 


Experi- 
mentals 


Control s 


Impact 


Experi- 
mental s 


Control s 


Experi- 

m^ntiil < 

iifc ■ 1 V a 1 9 




Loncurrcni 






















Avianva jod Lorps 


bl 


21.3% 


10. OV 


1L4 


75.1% 


78.5% 


-3.4 


$3,404.95 


$5,323.75 


-1,918.80* 


CET/San Jose 


152 


33.6 


25.9 


7.7 


81.4 


74.8 


6.6 


6 946 Of) 


fi 7nfi 74 


739.27 


Chicago Connons 


74 


6.0 


15.9 


-9.9 


82.7 


73.5 


9.1 


4 fill 41 


4 071 Aft 


759.63 


Lonnciicjr ^ritwSDurgn^ 


1 Qil 


55.6 


14.3 


4 1 . J*** 


74.3 


63.2 


11.1* 


1,969.75 


2,405.07 


-435.33 


East LA Skills Center 

erne /n^nuAM^ 
tuUj \uenverj 


100 


5.0 


9.0 


-4.0 


70.4 


67.8 


2.6 


4,509.77 


5,203.43 


-693.66 


183 


22.2 


12.9 


9.2 


65.6 


66.6 


-1.0 


2,998.48 


3,581.51 


-583.03 


Phoenix Job Corps 


130 


26.0 


11.1 


14.9** 


65.3 


73.1 


-7.8 


3,005.02 


3,575.39 


-570.37 


SER/Corpus Christi 


236 


55.4 


15.9 


3f^.6*** 


67.5 


71.2 


-3.8 


2,581.26 


2,643.74 


-62.48 
























LI LcnvrQ ^UQMas^ 




57.9 


17.1 


W • o 


77.2 


66.0 


11.2 


4,020.23 


3,538.57 


481.65 


LA Job Corps 


218 


19.4 


14.6 


4.8 


69.2 


64.6 


4.6 


4,389.13 


4,854.90 


-465.77 


Sequen tial/b roke red 






















Allentown (Buffalo) 


140 


30.3 


20.8 


9.5 


72.4 


73.9 


-1.5 


3,781.28 


3,299.92 


481.36 


8SA (NYC) 


119 


29.5 


29.0 


0.5 


72.1 


72.8 


-0.7 


5,362.25 


6,246.46 


-884.21 


CREC (Hartford) 


87 


22.7 


15.3 


7.4 


73.8 


69.3 


4.5 


5,024.72 


5,261.38 


-236.66 


P'Value of F-stati$tlc 








0.000*** 






0.724 






0.767 


All sites 


1.B39 


33.2 


16.4 


16.7*** 


72.0 


69.5 


2.5 


3,894.08 


4,098.81 


-204.73 



SOURCE: MDRC calculations from JOBSTART enrollment fom and survey data. 



MOTES: Calculations for this table uced data for all sample members for whom there were 24 months of follow-up survey data, including 
those with values of zero for outcomes and those who were assigned to JOBSTART but did not participate. 

Rando^.1 assignment did not always take place on the first of the month. For some sample members, the month of random assignment 
may be less than a month, beginning with the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means from two-way analysis of covariance procedures 
controlling for 19 kinds of difference in characteristics, other than site, before random assignment. The two categories used as factors were 
research assignment and site (see Ostle, 1975, p. 454). "All sites" outcomes are from a linear analysis of covariance procedure for the full 
sample controlling for up to 31 kinds of difference in characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and 
Appendix Table B.4). There may be slight discrepancies in reported sums and differences of these adjusted means because of rounding! n 

A two-tailed t-test was applied to each wi thin-site impact. An F-test was applied to the interaction between site and 
experimental or control status. The p-value of the F-stat ;tit is the probability that site impacts are different only because of random 
error. Statistical significance levels are indicated as *** « 1 percent; ** « 5 percent; * « 10 percent. 



Center (under both types of impact estimates) and at BSA in New York City (under the split- 
file approach). Sequential/brokered sites had the weakest educational attainment impacts, as 
one might expect from the generally higher control group receipt of a high school diploma or 
GED in these sites. This higher rate of control group receipt of a GED probably reflects the 
stronger interest in GED receipt among applicants in these sites, as discussed in an earlier 
section of this chapter, and the availability of alternative services in the two large cities of New 
York and Buffalo. El Centro in Dallas, SER/Corpus Christi, and Connelley in Pittsburgh's 
impacts were much larger than those for the other sites. There is a likely explanation for this 
that is related to program characteristics: These three sites placed great emphasis on GED 
attainment, in large part because of contractual provisions rewarding them for accomplishing 
this outcome. 

None of the individual site impacts on second-year employment were statistically 
significant in the split-file table and only one was in Table 6.4, but there did appear to be 
variation within each site category (although it v/as not statistically significant). In both tables, 
each of the three categories showed both relatively positive impacts and one or more sites with 
negative or close to zero impacts. In both tables, the employment rate impacts were still 
negative in six of the sites, although in three of these sites, the difference between controls 
and experimentals was very small (less than 2 percentage points). 

Impacts on second-year earnings showed more variation across sites than did the 
employment impacts; they also showed considerable variation within each of the three groups 
of sites. Because of the diverse earnings of youths within the individual sites and the small 
samples (owing to the fine level of disaggregation), only two of the 13 individual site impacts 
were statistically significant in Table 6.3 and only one was in Table 6.4. It is notable that there 
were positive and negative earnings impact estimates in each of the three groups of sites in 
both tables. 

Earnings impacts in Tables 6.3 and 6.4 differ substantially for a few sites in the 
demonstration. The adjustments used in TabL 6.4 make a larger difference for Chicago 
Commons than for all but one other site (the Phoenix Job Corps), complicating the comparison 
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of impacts for Chicago Commons with other sites. ^ According to Table 6.3 (the split-file 
table), which does not equalize measured baseline charact^*ri«nco across sites, the largest 
earnings impact estimate was $563.12 for El Centro in Dallas. Adjusting for site differences 
in sample characteristics, the El Centro earnings impact drops slightly to $481.65. Chicago 
Commons, in contrast, increases from a $77.56 impact estimate in Table 6.3 to $759.63, the 
largest earnings impact estimate in Table 6.4. This large change is attributable in part to the 
site's screening practices and training offerings, which led to unusual sample characteristics 
(especially a larger-than-average proportion of men).^^ Since Table 6.4 comes closer to making 
individual site impacts comparable, Chicago Commons should probably be viewed as producing 
positive, and relatively large, impacts. 

It is also noteworthy that three of the four sites with positive estimates in Table 6.4 were 
among those described above as having implemented the components of JOBSTART most 
successfully. Chicago Commons and CET/San Jose had the largest point estimates for second- 
year earnings impacts in Table 6.4, but it is not apparent whether this was due to their 
relatively concentrated doses of JOBSTART, the strength of their job placement efforts, their 
integration of education and training, their longstanding reputations in their communities, or 
other factors unique to these sites, such as their close ties to employers. On a sixth important 
program feature, they differed: As noted above, CET/San Jose served applicants without 
screening, while Chicago Commons imposed strict entry requirements.^^ 

The two other sites with positive earnings impacts in both tables (Allentown in Buffalo 
and El Centro in Dallas) operated different types of programs from CET/San Jose and Chicago 



impacts of the Phoenix Job Corps become less negative with the shift from split-file to adjusted 
impacts, but neither this change nor the one for Chicago Commons affects the basic conclusions of the 
chapter. Another important site, CET/San Jose, had the second largest earnings impact in both tables, 
chancing from $508.42 in Table 6.3 to $739.27 in Table 6.4. 

^-^It also related to the relatively small sample at Chicago Commons (74 people) and to unusual site- 
specific coefficient estimates for the regression adjustments for experimental-control differences ustd in the 
split-^file table. This change may also have been related to the small sample size, since in a small sample, 
outliers can have a strong influence on coefficient estimates. 

^^ese two sites also differed somewhat in their experimental-control difference in service receipt, 
with CET/San Jose, at 52 percentage points, having a larger gap than Chicago, at 32 percentage points (see 
Table 6.2). 
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Commons. ^ Allen town had a heavy education focus (with only one-third of the sample getting 
any training), no integration of education and training, weak job placement efforts, and the 
longest average length of stay in the program (nearly four months above the average). It also 
had the smallest experimental-control service difference of any site. El Centro's sequential/in- 
house program rested somewhere in between these two extremes in both program emphasis 
and length of stay. Its service receipt difference may have played a crucial role in the positive 
impacts, being the largest of any site (72 percentage points). 

Two sites assessed as implementing the JOBSTART program model well did not have 
positive earnings impacts in the second year of follow-up. Connelley in Pittsburgh did have 
strong educational attainment and employment rate impacts, but these had not yet translated 
into positive earnings impacts during the follow-up period. Connelley's long length of stay in 
the program (at 10 months, the second longest) may have been an important contributing 
factor. As to the second site, the Phoenix Job Corps, there is no clear reason for the lack of 
employment rate and earnings impacts. 

III. Suberoup Impacts^ bv Site 

This section continues the analysis by examining impacts on employment and earnings 
for key subgroups by site. At this fine level of disaggreption, the sample sizes in individual 
sites are so small as to make only the broadest conclusior s possible. In view of the tentative 
nature of this analysis, tables are presented in Appendix CI and the text presents the outlines 
of the findings. This analysis is based on site impacts adjusted for observed differences in pre- 
random assignment characteristics across the sites; that is, it uses the same approach as Table 
6.4. 

A. Second-Year Impacts for Men, b> Site 

The sample sizes for men at each site were usually less than half of the already small 
site sample sizes reported in Table 6.4. Since the full sample in each site yielded site 
differences in impacts that were not statistically significant, it is not surprising that most site- 
level impacts for men were also not statistically significant. 



A third site, SEFl/Corpus Christi, had g positive point estimate for the earnings impact according 
to the split-file method (Table 6,3) but not according to the adjusted method (Table 6.4) of calculating 
impacts. 
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With this caution in mind, the site story for men seems much the same as the full sample 
story. As in the full sample, employment and earnings impact differences across sites were not 
statistically significant, and not much should be made of them. However, the sites with 
substantial, positive point estimates for men were largely the same as the sites with positive 
estimates for the full sample. For example, CET/San Jose, Chicago Commons, and Allentown 
in Buffalo had point estimates of several hundred dollars for men, as was true of the full 
sample. However, there were two major exceptions to this pattern. First, El Centro in Dallas's 
impacts, which were positive for the full sample, were negative for men. Second, the largest 
estimate for impacts on earnings was for the 31 men at CREC in Hartford; the earnings impact 
in this site was driven by a 30 percentage point impact on employment. 

In view of the fairly negative results for the full sample of men reported in Chapter 5» 
it is encouraging that there were sites with positive impacts for men on second-year earnings. 

B. Second-Year Impacts for Women Living with Their O w n Child fren), by Site 

Even more than for men or other women, site impacts for this group were based on very 
small samples in some sites, virtually assuring that site impacts and differences in impacts were 
not statistically significant; for example, there were only 10 mothers at CET/San Jose. Again 
bearing in mind that these impact estmiates are shrouded in much uncertainty, the second-year 
labor market story for mothers seems a bit different from the stories for the full sample and 
for men. 

The largest of the earnings impact estimates, for El Centro in Dallas mothers, apparently 
is what drove the full sample impact for El Centro. This impact was large enough to be 
statistically significant. The next largest earnings impact, at the East Los Angeles Skills Center, 
was driven by an employment rate impact of 49 percentage points. Unlike the earnings impact 
for men at the Los Angeles Job Corps, the earnings impact at that site for women living with 
their own child(ren) was positive. The earnings impact of more than $500 at CREC in 
Hartford for women living with their own children, combined with a large positive estimate for 
men, implies that the full sample impact for CREC must have been driven by a large negative 
estimate for other women. The positive but small earnings impacts* among the 10 CET/San 
Jose mothers and the 63 Connelley in Pittsburgh mothers were driven by employment rate 
impacts of 50 percentage points and 22 percentage points, respectively. Earnings impacts in 
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the other sites were negative, down to a statistically significant estimate of -$3,084 at BSA in 
New York City, 

The findings for the three Job Corps sites for this subgroup hold special interest. Positive 
earnings impacts for women living with their own children, but not for other groups, at the 
Atlanta Job Corps and at the Los Angeles Job Corps contrast with earlier evaluation resulus 
for the residential Job Corps .20 The residential Job Corps was found to be quite effective for 
men and for women without child care responsibilities, but not for mothers. These preliminary 
findings seem to suggest that a nonresidential Job Corps program may be more appropriate for 
mothers. 

C. Second-Year Impacts fo r Other Women, bv Site 

The largest earnings point estimate was for the 14 "other women" (women who were 
not living with their own children) at Chicago Commons. Other positive impacts of more 
than $1,000 were achieved at the East Los Angeles Skills Center, the Los Angeles Job Corps, 
Connelley in Pittsburgh, and Allentown in Buffalo. El Centro in Dallas and CET/San Jose also 
had positive earnings impacts for these women. The other five sites had negative earnings 
impacts for these women during the second year. The biggest earnings loss for other women 
was at CREC in Hartford; thus, as noted aoove, it was these other women rather than mothers 
or men who were driving the negative overall earnings impact at CREC. 



IV. Summary and Future Impacts To Watch 

Despite strong policy interest in the influence of program structure, no simple story 
emerges. With respect to the full sample in each site, JOBSTART was sometimes effective 
and sometimes ineffective in yielding second-year labor market gains in brokered programs 
and in-house programs; it also showed varying success in both concurrent programs and 
sequential programs. This suggests that the JOBSTART program model can be implemented 
successfully in a variety of settings. 

Adjusting for site differences in sample characteristics, Chicago Commons and CET/San 
Jose had the largest positive estimates for second-year earnings impacts. Each integrated 
education into the training sequence more than did other sites, but each also had several other 



20See Mallar et al., 1982. 
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features that distinguished it from most other sites, including strong job placement and 
relatively intease concentration of many instruction hours into a I'ew months' length of stay. 
However, the two sites differed in the extent of initial screening of applicants, with CET/San 
Jose doing very little and Chicago Commons being among the sites with the most screening. 
Two other sites operating sequential/in-house and sequential/brokered progicims (each with a 
much stronger education emphasis, and one providing very little training) also produced positive 
earnings impacts in the second year of follow-up. 

The picture for subgroups is similarly mixed. Sites that had positive earnings impacts 
for the full sample generally had positive impacts for men and women considered separately. 
In contrast to negative findings from an earlier study of residential Job Corps mothers, 
JOBSTART mothers in two of three nonresidential Job Corps sites had earnings gains during 
the second year. These preliminary findings seem to suggest fl-it a nonresidential Job Corps 
program may be more appropriate for mothers than a residential program. 

Some JOBSTART sites (such as El Centro in Dallas) achieved both GED attainment 
gains and second-year earnings gains; others (the East Los Angeles Skills Center and at BSA 
in New York City) succeeded at neither. But the findings presented in this chapter provide 
evidence that sometimes there is a trade-off for program designer between emphasizing GED 
attainment and emphasizing accelerated occupational training for a specific job. In the two 
sites with the largest short-run adjusted earnings impacts - Chicago Commons and CET/San 
Jose - GED impacts were negative or fairly small. In the short run, emphasizing a specific 
skill may prove more effective in producing earnings impacts. However, the long-run picture 
could be quite different. 

Education can create new opportunities, and these may arise over a long period of time. 
In comparison, training for a specific job opens a few doors quickly, but in the long run, 
because of economic change and uncertainty, they may not be the right doors. The oldest 
members of t/ie JOBSTART sample were only 23 years old at the end of the follow-up 
available for this report; most sample members were considerably younger. Their transitions 
into full-time work and independent living arrangements will not be complete for several years. 
If the GED is important as a credential for better jobs or job training far into the future, labor 
market impacts at the two-year point may be quite misleading. 

In the final JOBSTART report, using four years of follow-up, it will be especially 
important to look at impacts at the end of follow-up. 
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APPENDIX A 
DATA SOURCES FOR THE EVALUATION 



Many data sources were used in this evaluation of the JOBSTART Demonstration. 
Baseline demographic data were collected at the time of random assignment. Management 
information system (MIS) data from the sites were used to measure participation hours. 
Twelve-month and 24-month follow-up surveys of applicants were conducted to measure 
impacts on experimentals (including those who did not participate) compared to controls; the 
impacts concerned amounts of education and training received, employment and earnings, and 
other outcomes. The 12-month survey also dealt with the experiences of participants in the 
JOBSTART program. Much qualitative inforraation, including interviews with program staff 
as well as focus groups and in-depth interviews with participants, was used in conjunction with 
the quantitative information. Each data source is described below. 

L JOBSTART Enrollment Forms 

The JOBSTART Enrollment Form, designed by MDRC and filled out by program staff 
at the time of random assignment, was the major source of information about the demographic 
and socioeconomic characteristics of sample members. It included data on age, sex^ ethnicity, 
family composition, educational attainment, and time since dropping out of school, as well as 
basic information on welfare and employment histories. The enrollment form was completed 
for all but one sample member.^ 

IL JOBSTART Management Information System Forms 

Sites used a number of ^ IDRC-designed forms to report on the progress of participants 
in JOBSTART. The most important of the;se were: 



^This sample member was excluded from the impact analysis, since all demographic variables from 
the enrollment form are missing. For many of the sample members, a few specific pieces of demographic 
information are missing. In the impact analysis, the predicted values based on similar sample members 
were substituted for these missing observations. 
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A. Monthly Participation Report 

The Monthly Participation Report provided the number of hours that participants spent 
in basic education, occupational skills training, or other kinds of JOBSTART activities each 
month. It also provided information on the type of occupational skills training in which 
participants in training enrolled. Sites reported actual hours attended, not the number of hours 
scheduled. 

Participation data used in this report were collected from August 1985 - the beginning 
of random assignmv-?nt - through January 1989. The month of random assignment was 
included as a month of follow-up for participation, although the participant may hax'e been 
randomly assigned late in the month. Those assigned in the last month of ranaom assignment 
- November 1987 - had 15 months of follow-up participation data. The vast majority of 
the sample had at least 24 months of follow-up. 

Collecting strictly comparable data across sites was not always possible, for two reasons: 
First, the services provided in each site varied; second, there was some inconsistency in the way 
sites reported hours for activities other than basic education or occupational training classes. 
For example, a number of sites supplemented education and/or training classes with formal 
classroom instruction in a variety of topics generally termed "life skills." Some sites reported 
these as education hours; others counted them as training hours. In order to have similar 
definitions of the basic components - education and training - MDRC modified the reported 
hours at sites, so that time spent in such activities as life skills classes was counted under "other 
activities."^ The education hours reported by CET/San Jose also were adjusted to reflect only 
hours spent in the site's GED class.^ 



sites were El Centre in Dallas, the Los Angeles Job Corps, the Atlanta Job Corps, and 
Allentown in Buffalo. At El Centro, one-half of all education hours prior to December 1986 were spent 
in life skills. After 1986, one-fourth of the reported education hours were spent in life skills. The hours 
were counted as hours in "other activities" by MDRC. At the Los Angeles Job Corps, participantf spent 
one-half of their reported education hours in activities such as art, gym, and "world of work" for the first 
three months after enrollment. MDRC moved one-half of the education hours to hours in "other 
activities" for those months. At the Atlanta Job Corps, 10 hours each week were spent in activities such 
as life skills, driver education, and health. MDRC moved 28.6 percent of the reported education hours 
to hours in "other activities." Allentown in Buffalo included such hours in its reported occupational 
training hours. MDRC moved all reported occupational training hours that did not have an associated 
type of training to hour* in "other activities." 

^CET/San Jose reported 30 percent of each participant's occupational training hours as education, 
which included time spent on training-related basic skills in occupational training courses as well as hours 
in the site's GED class. For consistency with other sites, the education and training hours at CET were 

(continued...) 
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Other differences remained, however A number of sites offered limitexl amounts of work 
experience as part of the JOBSTART program. Some sites reported these hours as training 
hours; others reported them under "other activities.""^ No adjustments were made in these 
hours. Finally, the Phoenix Job Corps did not report hours spent by participants in life skills 
or avocational activities, although the other two Job Corps sites did. 

Appendix Table Al shows the common elements and variations in component activities 
across sites. In general, participation hours reported as being in the education component 
consisted of time spent in classes devoted to basic education or GED preparation; they did not 
include work on training-related basic skills done in occupational training courses. In all sites, 
participation hours that were counted in the training component included all activities offered 
in occupational training curricula, including units on training-related educational skills (such as 
Business English or Business Math) and employability development (instruction in work 
behaviors and job search). In the following sites, the hours counted as training also included 
time spent in work experience or on-the-job training: Connelley in Pittsburgh, El Centro iii 
Dallas, the Phoenix Job Corps, and the Los Angeles Job Corps. Hours spent in "other 
activities" varied considerably acioss sites and included instruction in life skills, work experience, 
and orientation and avocational activities. 

In order to assess the quality and completeness of the participation data, MDRC staff 
reviewed the teachers* class attendance records and other sources of data for a randomly 
selected sample of participants. For the most part, there was agreement between hours found 
in teachers' records and the Monthly Participation Reports. If more than 20 percent of the 
cases in a quality control sample had discrepancies greater than 10 percent between site- 
reported hours and hours obtained in the check, MDRC scheduled either a re-collection of the 
data or retraining of site staff, depending on the seriousness of the discrepancies.^ 



^(...continued) 

recalculated by MDRC, and only hours spent in the separate GED class were included as education hours 
in this report. 

"^At EGOS in Denver, hours spent by participants in Vork study** were not reported. 

^Because it was necessary to obtain records from a number of service providers, many of which did 
not maintain complete records for long periods, occupational training hours in brokered sites were the 
most difficult to confirm and probably have the greatest variation between actual and reported hours. The 
difficulty MDRC staff had in obtaining and verifying date from training providers reflects the difficulty sites 
had in monitoring hours for participants once they were no longer in the site. Problems were found even 
in the two sites with the best data from service providers: One site apparently over-reported hours, while 
one site apparently under-reported hours. Because the number of participants who entered trrining in 

(continued...) 
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TABLE A.l 



ACTIVITIES INCLUDED IN PARTICIPATION HOURS, 
BY SITE AND COMPONENT 



Site 


Education 


Training 


Other Activities 


All sites 


Classes in basic 
education or GED 
preparation 


Classroom occupational 
skills training, 
including classes in 
training-related basic 
skills and employa- 
bility development 


Varies 


Except! onsp by site 








Allentown (Buffalo) 


a 




Life skills* 


Atlanta Job Corps 






10-day orientation, 
work experience and on- 
the-job training, life 
skills and avocational 
activities* 


BSA (NYC) 


May include a few hours 
per week in computer- 
assisted life skills 
curriculum 




Life skills 


CET/San Jos^ 


a 


a 


None 


Chicago Commons 






None 


Connelley 
(Pittsburgh) 




Work experience 
mentorships 


None^ 


CREC (Hartford) 


Includes soire hours in 
employability develop- 
ment activities 




Work experience 
internships 


East LA Skills 
Center 






None 


EGOS (Denver) 






None 


El Centro (Dallas) 


a 


Work experience 
internships 


Life skills* 


LA Job Corps 


a 


Work experience and on-- 
the-job training 


5-day orientation, life 
skills and avocational 
activities* 


Phoenix Job Corps 




Work experience and on- 
the-job training 


8-day orientation^ 


SER/Corpus Christi 






None 



SOURCE: Adapted from Auspos et al.» 1989. 



NOTES: *Reported hours were adjusted by MDRC. 

^Site did not report participation in a one-hour after-school component consisting of 
counseling and other supijort services in school year 1986-87. 

^Site did not report participation hours in work-study positions. 

^Site did not report participation hours in life skills and avocational activities. 
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B. Other Man agement Information System Data 

As part of the monthly monitoring system, sites also reported on the end-of-month status 
of each participant, the participants who had been terminated and the reason for termination, 
and job placement and GED receipt among participants. The follow-up surveys proved to be 
a more complete source of data for employment and GED receipt, since they included activity 
by experimentals that might not have been reported to site operators as well as the experiences 
of the control group. Consequently, the sun^eys are the only source of these data used in this 
report. 

ni. Test of Adult Basic Education 

The Test of Adult Basic Education (TABE), a modification of the California Achievement 
Test, was used to measure reading levels of experimentals. Prior research has shown the test 
to be a reliable and valid measure of reading ability. The test was used at two points in time: 
shortly after random assignment (as a baseline measure)^ and after participants had spent some 
time in the program (usually after about 100 hours of education), as a measure of reading level 
gains.^ 

About 20 percent of the total experimental sample did not take a baseline TABE. The 
percentage tested varied by site from a high of 100 percent to a low of 42 percent. The Job 
Corps sites and CET/San Jose had the lowest percentage of experimentals with baseline 
TABEs. 



^(...continued) 

sequential/brokered sites was small, the misreporting of training hours did not greatly affect the averane 
hours of trammg presented in the report. b j ^ mc avcidyc 

.r^J^^.^Z f^' ^h^TABE was also used as a test of reading-level eligibility and consequently was 
fn r S 1?" n f ^.'H'erimentals. A number of other riding tests were t^dminiJteTed 

in the other sites. Data from these sites were not included in the analysis because the data were not 

SlleT T^eZIT' ^"^^'"'^ " ^-^""^ — — 

^The actual number of hours of education between random assignment and the first follow-un test 
varied a,ns.derably because of differences in measuring hours of education and delays in admlnisS th 
T'y. : I" ^'"^ °^ demonstration, sites were asked to test every three months 

which resulted in considerable variation in the number of hours after which participants were lesle" 
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IV. Fnllnw.U p Survevs 

■ Eighty percent (1.839) of the 2.311 sample members randomly assigned to the 
experimental or control group between August 1985 and November 1987 were interviewed. 
(Table A.2 presents survey response rates by site.) These 1.839 youths constituted the sample 
for this report, and each of them provided follow-up information for 24 months after the date 
of his or her random assignment. Most responded to both the 12- and 24-month follow-up 
surveys (1.604 or 87 percent of responders). while the remainder responded to a special 
combination survey covering the entire 24 months, which was fielded for youths who did not 
respond to the 12-month survey but were located at 24 months. The surveys were conducted^ 
either in person or. for the approximately one-fifth of the sample who had moved out of the 
area, by telephone, one and two years after random assignment. The interviews lasted about 
45 minutes and provider^ nformation about the applicant's experience during the period of 
follow-up covered in that survey wave. Respondents were asked about their employniem 
history, family status, welfare receipt, and receipt of education or training outside of 
JOBSTART. During the 12-raonth survey, experimental who did not participate in 
JOBSTART were asked why; participants were asked what they like and disliked about the 
program and their reasons for leaving. (Appendix B discusses issues of sample bias and data 

quality for the survey.) 

Sample members who could be located were generally willing to be interviewed. Some 

could not be located while others simply could not be contacted. 

Some completed surveys lacked some information that was important in calculating 
impacts. Because the presence of missing data might have been correlated with an observed 
or unobseived prior attribute, dropping cases with missing data from the analysis might have 
biased the impact estimates or produced month-to-month inconsistencies. Imputing values is 
possible using a procedure that does not bias results. A separate regression was run for each 
variable with missing values, yielding predictea values for the missing data. These predicted 
values were used as estimates of the missing values. Continuous outcomes may contain outliers 
- extreme values that overly infiuence estimates. In the analysis, these were treated as 
missing, and the usual procedures for missing values were applied. 



8MDRC contracted with Abt Associates, a Boston-based survey firm, to ^"^Ple'"^ ^' /"^"^f ' .'"J 
monitor the su^ey. Completed surveys were data-entered and checked for completeness by Abt. Members 
of the Abt staff also assisted in the design of the survey instrument. 
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TABLE A. 2 

RESPONSE RATES FOR 24-MONTH SURVEY, BY SITE 



Site Sample Size Responded Did Not Respond 
Concurrent 

Atlanta Job Corps 80 76.3% 23.8% 

CET/San Jose 200 76.0 24.0 

Chicago Commons 93 79.6 20.4 

Connelley (Pittsburgh) 219 84,0 16.0 

East LA Skills Center 126 79,4 20.6 

EGOS (Denver) 237 77.2 22.8 

Phoenix Job Corps 153 85.0 15.0 

SER/Corpus Christi 300 78,7 21.3 

Sequential /in-house 

El Centro (Dallas) 200 77.5 22.5 

LA Job Corps 296 73.6 26.4 

Sequential /brokered 

Allentown (Huffalo) 147 95.2 4.8 

BSA (NYC) 151 78,8 21.2 

CREC (Hartford) 109 79.8 20.2 



SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all 2,311 sample members. 
Rows may not total 100.0 percent because of rounding. 
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V. Qualitative Data 

Qualitative descriptions of the program and participants' experiences in it were obtained 
from a variety of sources and were used to complement the analysis of the quantitative data. 

MDRC research staff visited sites and conducted structured interviews with program 
administrators, counselor/coordinators, and teaching staff to determine recruitment practices, 
the content of services in the education and training components, job placement and other 
activities, the range of support services and retention strategies, and staffing patterns and staff 
experience with JOBSTART. Staff also observed education and training classes in each site 
and visited some of the organizations that provided occupational training to JOBSTART 
participants in the sequential/brokered sites. Sites were typically visited by research staff once 
during the early phase of the demonstration and twice in the second year of program 
operations. This information was supplemented by ongoing reports on program operations and 
classroom observations provided by MDRC operations staff, who visited each site at regular 
intervals: once every month in year 1 and once every two months in year 2 of the operational 
period. (Interviews and observations concerning the education component were developed in 
conjunction with an education expert, who worked with MDRC as a consultant*) 

Information about participants* reactions to JOBSTART was obtained from focus group 
discussions with 46 JOBSTART participants in four sites between May 1987 and February 
1988. Female participants were interviewed at Connelley in Pittsburgh and at BSA in New 
York City; men were interviewed at EI Centro in Dallas and at the Los Angeles Job Corps. 
Each session was attended by between 9 and 14 participants and lasted between two and two 
and a half hours. At Connelley and El Centro, the groups were made up of participants in 
attendance on the session day; at the Los Angeles Job Corps, staff selected students who were 
doing well in the program; the BSA group included both current participants in education 
and women who had already moved on to occupational skills training. Because they included 
many participants who stayed longer than the average and/or were doing well in the program, 
the groups were not representative of all JOBSTART participants. Nevertheless, used in 
conjunction with the survey responses, the focus group discussions provided valuable insights 
into participants' expectations about the program, what helped and hindered their participation, 
their opinions of the education and training components, and their recommendations for 
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improving the program. MDRC hired consultants to develop the discussion topics, moderate 
the groups, and analyze the responses. 

A series of in-depth interviews was conducted by another consultant with 15 JOBSTART 
participants in four other sites (CREC in Hartford, EGOS in Denver, Allentown in Buffalo, 
and the Atlanta Job Corps) between November 1986 and September 1987. These profiles 
provided additional, although impressionistic, information about the lives of some JOBSTART 
participants prior to and during the demonstration. The report also drew on the observations 
of JOBSTART staff and selected participants who attended a conference on Youth 
Employment Initiatives, sponsored by MDRC, in October 1987.^ 



^Sec Manpower Demonstration Research Qjiporation, 1988, for a summary of the conference 
discussions. 

-193- 



APPENDIX B 

IMPACTS OF JOBSTART: METHODOLOGICAL ISSUES 



As outlined in Chapter 2, several methodological issues had to be addressed to answer 
the key evaluation questions. 

L Selection Bias 

Did random assignment succeed in creating a group of JOBSTART controls with the 
same pre-program characteristics as JOBSTART experimental? If sample members become 
experimentals or controls completely at random, there are no systematic measured or 
unmeasured differences between the two groups before program treatment. Under those 
circumstances, average outcomes among controls measure what average outcomes would have 
been among experimentals had the treatment not been available to them, and the difference 
in average outcomes between experimentals and controls measures the program's effect. If 
there are systematic preexisting differences between experimentals and controls, then measured 
differences in post-treatment outcomes confound true program effects with biases due to the 
selection of more people from some groups to be experimentals and more people from other 
groups to be controls. 

Table B.l presents, one at a time, average characteristics for experimentals, controls, and 
both groups together. There were only slight differences between groups in a few individual 
characteristics, and no overall pattern of systematic differences between groups. 

An alternative, more rigorous way to deal with this issue is to use linear regression 
analysis. To implement statistical tests for systematic experimental-control differences in those 
characteristics used in impact regressions. Table B.2 presents linear regression results measuring 
the extent of selection bias for the 2,311 members of the JOBSTART sample who filled out 
enrollment forms. ^ The first column of Table B.2 shows the same slight differences in 
individual characteristics and the >;ame absence of systematic differences as did Table B.l. The 
final entry in the column, the p-valvic of the F-statistic, is veiy close to one, providing strong 



^One sample member who did not wrnpleie an cnroliniera form was excluded from the impact 
analysis. 
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TABLE B.l 

CHARACTERISTICS AT THE TIMK OF RANDOM ASSIGNMENT, BY RESEARCH STATUS 





Sampl e 






Both 




Characteristic and Subgroups 


Size 


Experimentals 


Controls 


Groups 


P 














Unmon 


968 


53.8% 


51.3% 


52.6% 


0.284 


1 1 wM 


871 


46.2 


48.7 


47.4 




Ethnicity 












WhitBi non-Hispanic 


155 


7.6 


9.3 


8,4 


0.340 


R1 Ark nnn-HisnAnir 


840 


46.3 


45.1 


45.7 




Hi SDani c 


783 


43.3 


41.8 


42.6 




Other 


61 


2.8 


3.8 


3.3 




Ethnicitv. bv Qender 












Women 












White. non-Hisoanic 

VP 9 1 1 V W A II W II 1 1 1 «# mm Vi 1 1 1 W 


84 


4.3 


4.8 


4.6 


0.637 


Black non-Mi s D^^ni r 


445 


25.2 


23.1 


24.2 




Hi SDani c 


413 


23.0 


21.9 


22.5 




other 


26 


1.4 


1.5 


1.4 




Men 












Whitp non-HisDanic 


71 


3.3 


4.5 


3.9 




Rlark non-HisDanic 


395 


21.1 


21.9 


21.5 




Hi^Dflnic 


370 


20.3 


19.9 


20.1 






35 


1.6 


2.4 


1.9 




Parpntfll status 












Unmon llVinn with 












own child(ren) 












No 


484 


27.5 


25.1 


26.3 


0,311 


Yes 


484 


26.3 


26.3 


26.3 




Mon u/tin hxuo nwn child (I^Pni 












No 


765 


39.8 


43.5 


41.6 




Yes 


106 


6.3 


5.2 


5.8 




Employed within past year 












No 


870 


47.2 


47.4 


47.3 


0.929 


1 es 


969 


52.8 


52.6 


52.7 




Prior employment, by gender 












Women employed within 












pod L jr CO 1 












No 


547 


29.7 


29.8 


29.7 


0.538 


Yes 


421 


24.1 


21.6 


22.9 




Men employed within 












pdst yfc^r 












No 


323 


17.5 


17,6 


17.6 




Yes 


548 


28.7 


31.0 


29.8 




Sample size 


1,839 


949 


890 







(continued) 
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TABLE B.l (continued) 



Sample Both 
Characteristic and Subgroups Size Experimental s Controls Groups p^ 



Left school in grade 11 or 12 



No 1,078 
Yes 761 

Received occupational 

training within past year 
No 1,529 
Yes 310 

Age 

16-19 1,359 
20 or 21 480 

Marital status 
Ever married 174 
Never married 1,665 

Living in own household or 

with boy/girlfriend 
No 1,500 
Yes 339 

Own AFOC case or receiving 

General Assistance 
No 1,344 
Yes 495 

Own AFDC case 
No 1,446 
Yes 393 

Receiving Food Stamps 
No 1,143 
Yes 696 

Arrested since age 16 
No 1,567 
Yes 272 

Lived with both parents at 
age 14 

No 1.198 
Yes 641 



57.7% 59.6% 58.6% 0.432 

42.3 40.4 41.4 

84.0 82.2 83.1 0.321 

16.0 17.8 16.9 

74.1 73.7 73.9 0.857 
25.9 26.3 26.1 

9.3 9.7 9.5 0.775 

90.7 90.3 90.5 

82.4 80.7 81.6 0.340 
17.6 19.3 18.4 

74.0 72.1 73.1 0.375 

26.0 27.9 26.9 

79.2 78.0 78.6 0.509 

20.8 22.0 21.4 

62.0 62,4 62.2 0.860 

38.0 37.5 37.8 

84.8 85.6 85.2 0.633 

15.2 14.4 ]4.8 

66.0 64.3 65.1 0.446 

34.0 35.7 34.9 



Sample size 1,839 949 890 



(continued) 




TABLE B.l (continued) 



Characteristic and Subgroups 


Sample 
Size 


Experimental 


Controls 


Both 
Groups 


Site 










Concurrent 










Atlanta Job Coros 


61 


3.5% 


3.1% 


w 3% 1.000 


CET/San Jose 


152 


7.9 


8.7 




Chicago Coimons 


74 


3.9 


4.2 


4.0 


Connelley (Pittsburgh) 


184 


9.9 


10.1 


10.0 


East LA Skills Center 


100 


5.4 


5.5 


5.4 


EGOS (Denver) 


183 


9.8 


10.1 


10.0 


Phoenix Job Corps 


130 


7.1 


7.1 


7.1 


SER/Corpus Christi 


236 


12.5 


13. 1 


12.8 


Sequent i a1 /in-house 










El Centro (Dallas) 


155 


8.5 


8.3 


8.4 


LA Job Corps 


218 


12.1 


11.6 


11.9 


Sequential /brokered 










AUentown (Buffalo) 


140 


7.8 


7.4 


7.6 


BSA (NYC) 


119 


6.8 


6.1 


6.5 


CREC (Hartford) 


87 


4.7 


4.7 


4.7 


Sample size 


1.839 


949 


890 





SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 



NOTES: Calculations for this table used data for all 1,839 sample members for whom 
there were 24 months of follow-up survey data. Sample sizes reported may fall short of this 
number because of items missing from some sample members' questionnaires. 

Distributions may not tot2»l 100.0 percent because of rounding. 

*The column labeled "p" is the statistical significance level of the 
difference in distributions of characteristics between groups: that is, p is the probability 
that observed proportions in each subgroup differ by research status only because of random 
error. A Pearson chi-square statistic was used to test the hypothesis of equal 
distributions. Statistical significance levels are indicated as = 1 percent; = 5 
percent; * « 10 percent. 
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TABLE B.2 



ESTIMATED REGRESSION COEFFICIENTS FOR THE PROBABILITY OF 
ASSIGNMENT TO THE EXPERIMENTAL GROUP 



24-Month Combination-Survey 

Regressor or Statistic Full Sample Impact Sample Responders 

0.503*** 0.516*** 0.464*** 

(0.010) (0.012) (0.032) 

^'connelley (Pittsburgh) -0.001 0.001 -0.077 

(0.054) (0.060) (0.175) 



Constant 



CET/San Jose 

SER/Corpus Christi 
EGOS (Denver) 

Chicago Conrions 



BSA (NYC) 



CREC (Hartford) 
Phoenix Job Corps 
East LA Skills Center 
LA Job Corps 
Atlanta Job Corps 
Male 



.0.004 -0.026 0.081 

(0.049) (0.056) (0.177) 



-0.000 0.002 0.083 

(0.045) (0.052) (0.176) 

0.000 -0.003 -0.152 

(0.065) (0.074) (0.188) 



El Centro (Dallas) -0.003 0.026 0.258 

(0.051) (0.058) (0.215) 



0.002 0.054 0.207 

(0.05A) (0.062) (0.152) 



Allentown (Buffalo) 0.031 0.042 0.175 

(0.05C) (0.063) (0.175) 



0.003 0.012 -0.051 

(0.059) (0.067) (0.156) 

0.000 0.015 0.211 

(0.053) (0.058) (0.368) 

0.007 0.003 0.501** 

(0.055) (0.062) (0.2^"^ 

-0.006 0.024 0.192 

(0.047) (0.054) (0.134) 

-0.001 0.044 -0.176 

(0.069) (0.079) (0.290) 

-0.048* -0.050 -0.042 

(0.027) (0.031) (0.091) 



(continued) 
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TABLE B.2 (continued) 



Regres sor o- Statistic 
White, non-Hispanic 



Hispanic 



Other ethnicity 



Age 20 or 21 

No phone number on 
enrollment form 



Male parent 

Female parent living 
with own child(ren) 



Limited English 



Arrested since age 16 



Convicted since aqe 16 



Own AFDC case 



Receiving Food stamps 



Never married 



Household AFDC case 



Receiving Medicaid 



Left school in grade 11 or 12 



Full Sample 

-0.023 
(0.042) 

0.018 
(0.030) 

-0.014 
(0.068) 

0.010 
(0.026) 

-0.063 
(0.050) 

0.078 
(0.047) 

0.008 
(0,035) 

-0.013 
(0.057) 

0.049 
(0,037) 

-0.059 
(0.054) 

-0.012 
(0.037) 

-0.020 
(0.030) 

-0.010 
(0.040) 

0.069** 
(0.033) 

-0.030 
(0.031) 

0,013 
(0.023) 



24-Month 
Impact Sample 

-0.040 
(0.048) 

0.028 
(0.03E) 

-0.073 
(0.076) 

-0.002 
(0.029) 

-0.059 
(0.057) 

0.076 
(0.054) 

-0.002 
(0.039) 

0.048 
(0.066) 

0.036 
(0.042) 

-0.023 
(0.063) 

-0.008 
(0.041) 

-0.007 
(0.034) 

0.019 
(0.044) 

0.070* 
(0.037) 

-0.027 
(0.035) 

0.024 
(0.025) 



Combination-Survey 
Responders 

-0.352** 
(0.166) 

0.072 
(0.097) 

-0.034 
(0.162) 

-0.063 
(0.088) 

-0.265* 
(0.148) 

0.135 
(0.134) 

0.123 
(0.113) 

-0.076 
(0.155) 

0.109 
(0.102) 

-0.186 
(0.181) 

0.016 
(0.122) 

-0.027 
(0.090) 

0.090 
(0.132) 

0.010 
(0.112) 

0.043 
(0.099) 

-0.016 
(0.075) 



(continued) 
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TABLE B.2 (continuod) 



Regressor or Statistic 


Full Sample 


z*t-Montn 
Impact Sample 


LOiilul nfltiuri'"0ui vcjr 

Responders 


Lived with both parents 
at age 14 


-0.020 
(0.023) 


-0.012 
(0.026) 


-0.141* 
(0.082) 


Employed within past year 


(0.023) 


0 OOR 
(0.025) 


0.048 
(0.074) 


Number of 
observations 


2.311 


1,839 


235 


Number of 
experimental 


1,162 


949 


109 


Number of controls 


1.149 


890 


126 


Degrees of freedom 
for error 


2,2/9 


1 fin? 


203 


Error mean square 


0.251 


0.252 


0.245 


R square 


0.008 


0.009 


0.148 


Mean of dependent 
van ao i e 


0.503 


0.516 


0.464 


F-statistic 


0.596 


0.538 


1.140 


P-value of F-statistic 


0.963 


0.983 


0.291 



SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: The dependent variable in each regression equation was unity for each 
experimental and zero for each control. Each characteristic on the right-hand side of each 
equation was measured as a deviation from its mean. The standard error of each coefficient 
estimate is enclosed in parentheses. ■ -i 

A two-Jailed t-test was applied to each coefficient estimate. Statistical 
significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 

The p-value of the F-statistic is the probability of obtaining these 
coefficient estimates if the true chance of becoming an experimental did not vary with any 
characteristic. Thus, the closer the p-value is to unity, the more successful was random 
assignment in equating average characteristics of experimental and controls. 
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evidence that there was no overall pattern of differences between experimentals and contiols. 
It shows that random assignment created two groups >vithout systematic overall differences in 
characteristics before enrollment. There were statistically significant differences in only two 
individual characteristics. For the full sample, experimentals were slightly less Ukely to be male 
and more likely to live in a household with someone else who received AFDC. 

Among the 1,839 survey responders (see column two of Table B.2) and among the 235 
combination survey responders (column three), the results of random assignment were similar. 
Although, judging from the statistically insignificant p-values for the survey responders, there 
were no systematic overall differences, the experimental survey responders were slightly more 
likely to live in a household with someone else who received AFDC. Combination survey 
responders were less likely to be white non-Hispanic, slightly less likely to have no phone 
number on their enrollment form, and slightly less likely to have lived with both parents at age 
14. They were, however, more likely to have been at the East Los Angeles Skills Center. 

The procedure used to calculate all the impacts presented in this report took these slight 
differences in characteristics into account, and estimated the impacts that would have occurred 
had these slight differences not existed. 

11. Nonresponse Bias 

Were those sample members for whom there are continuous data for 24 months 
representative of the full JOBSTART sample, including nonresponders? A high degree of 
mobility among disadvantaged young dropouts makes it difficult for survey interviewers to locate 
all of them a year or two after they have been enrolled into a research sample. As noted in 
Appendix A, 1,839 of the 2,311 full sample members furnished data covering 24 months, either 
at both the 12-month and 24-month junctures or at the 24-month juncture, for an overall 
response rate of 79.6 percent (81.7 percent for experimentals and 77.5 percent for controls).^ 
See Table A.2 for site-specific information on response rates. 

There were system- tic differences in characteristics between those who responded to the 
surveys and those who did not respond. Table B3 presents linear regression results measuring 
the extent to which average characteristics for the 1,839 survey responders differed from 



^There are two types of nonresponse. Unit nonresponse is the failure to ascertain answers to any of 
the questionnaire items. Item nonresponse is the failure to obtain only some answers. All the response 
rates mentioned here are unit response rates. 
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TABLE B.3 



ESTIMATED 


REGRESSION COEFFICIENTS FOR THE 
UNIT SURVEY RESPONSE 


PROBABILITY OF 


Regressor or Statistic 


Sample and Deoendent Variable 
24-Month Impact Combination Survey 
Sample Unit Survey Unit Survey 
Response Response 
1,839/2,311 235/1,839 


Constant 


0.796*** 
(0.008) 


0.128*** 
(0.008) 


Experimental status 


0.040** 
(0.017) 


-0.028* 
(0.015) 


01 le 

Connelley (Pittsburgh) 


0.048 
(0.043) 


-0.023 
(0.039) 


SER/Corpus Christi 






CET/San Jose 


-0.016 
(0.039) 


-0.060* 
(0.036) 


EGOS (Denver) 


-0.028 
(0.036) 


-0.056* 
(0.034) 


Chicago Commons 


0.011 
(0.051) 


0.039 
(0.C48) 


El Centro (Dallas) 


-0.014 
(0.040) 


-0.084** 
(0.038) 


BSA (NYC) 


0.007 
(0.043) 


0.064 
(0.040) 


Allentown (Buffalo) 


0.168*** 
(0.046) 


0.030 
(0.041) 


CREC (Hartford) 


0.017 
(0.047) 


0.065 
(0.044) 


Phoenix Job Corps 


0.061 
(0.042) 


-0.127*** 
(0.038) 


East LA Skills Center 


0.,025 
(0.044) 


-0.065* 
(0.040) 


LA Job Corps 


-0.061 
(0.037) 


0.129*** 
(0.035) 


Atlanta Job Corps 


-0,034 
(0.055) 


-0.054 
(0.051) 



(continued) 
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TABLE B.3 (continued) 



Sample and Dependent Variable 
24-Month Impact Combination Survey 



Sample Unit Survey Unit Survey 
Response Response 
Regressor or Statistic 1,839/2,311 235/1,839 



Male 


-0.027 


O.OJb'^ 




(0.022) 


(0.020) 


White, non-Hispanic 


-0.037 


A A 1 O 

-O.Olo 




(0.034) 


(0.031) 


Hispanic 


-0.013 


A AO O 
0.0^^ 




(0.024) 


(0.023) 


Other etnmciiy 


0 • ODl 


All Cifit 

0 . 110 




(0.054) 


(0.049) 


Age 20 or 21 


0.001 


A A yi c 

-0.046** 




(0.020) 


(0.019) 


No phone number on enrollment form 




0.04 1 




(0.040) 


(0.037) 


Male parent 


A AO A 

-0.029 


A A C 1 

O.Obl 




(0.038) 


(0.035) 


remaie parent living with own cniiairen; 




\Jt\JCC 




(0.028) 


(0.025) 


Limited hnglish 


A AOQ 

-O.Ojo 


0.01 J 




(0.045) 


(0.042) 


Arrested since age 16 


-U. UU^ 


A 079*** 
U .0/ 




(0.029) 


(0.027) 


Convicted since age 16 


-0. oyb" 


A A7 1 it 

-0 .0/ 1 




(0.043) 


(0.041) 


Own ArUL case 




-U . UID 




(0.029) 


(0.026) 


Keceivi ig rood oiamps 


u • uu/ 


u . UUl 




(0.024) 


(0.022) 


Never married 


-0.008 


0.002 




(0.032) 


(0.029) 


Household AFDC case 


-0.014 


-0.005 




(0.026) 


(0.024) 


Receiving Medicaid 


-0.003 


-0.002 




(0.025) 


(0.023) 



(continued) 
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TABLE B.3 (continued) 



Regressor or Statistic 



Sample and Dependent Variable 
24-Month Impact Combination Survey 
Sample Unit Survey Unit burvey 
Response Response 
1,839/2,311 235/1,839 



Left school in grade 11 or 12 


0.028 


-0.035** 




(0.018) 


(0.016) 


Lived with both parents at age 14 


0.050*** 


-0.007 




(0.019) 


(0.017) 


Employed within past year 


0.011 


-0.012 




(0.018) 


(0.016) 


Number of observations 


2.311 


1,839 


Number of experimental 


1.162 


949 


Number of controls 


1,149 


8S0 


Degrees of freedom for error 


2,278 


1.806 


L. 1 1 U 1 lliCQII dUUGl C 


n 1 r n 
0. 159 


0. 106 


R square 


0.034 


0.070 


Mean of dependent variable 


0.796 


0.128 


F-statisti c 


2.482 


4.214 


P-value of F-statistic 


0.000 


0.000 



SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: The dependent variable in each regression equation was unity for survey 
response or combination-survey response and zero otherwise. Each characteristic on the 
right-hand side of each equation was measured as a deviation from its mean. The standard 
error of each coefficient estimate is enclosed in parentheses. 

A two-tail2d t-test was applied to each coefficient estimate. Statistical 
significance levels are indicated as *** « 1 percent; ** « 5 percent; * « 10 percent. 

The p-value of the F-statistic in column one or column two is the probability 
of obtaining these coefficient estimates if the true chance of responding to the survey or 
to the combination survey did not vary with any characteristic. Thus, the closer the 
p-value is to zero, the more important are differences in characteristicr, between survey 
responders and nonresponders or between combination-survey responders and those who 
responded to only one of the two surveys. 
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average characteristics at random assignment for the 472 nonresponders. Since the final entry, 
the p-value of the F-statistic, is zero to three decimal places, there is strong evidence of 
systematic differences between responders and nonresponders. Responders were significantly 
less likely to have been convicted between the age of 16 and the time of random assignment, 
and significantly more likely to have lived with two parents at age 14. Also, better response 
was found at Allento^^m in Buffalo, even after taking differences in individual characteristics 
into account. In addition, responders were somewhat more likely to be experimentals than 
controls. 

When nonresponse is randomly distributed among members of both treatment and control 
gro ips, it is troublesome only because it reduces the sample size and thus the statistical power 
to find impacts of a given size. Randomly distributed nonresponse does not alter the expected 
values of adjusted mean outcomes, and thus does not bias impacts. However, when 
nonresponse is greater among one research group (such as controls) or among members of 
either research group with certain characteristics (such as men), impacts may be biased slightly 
unless corrected for nonresponse. The most flexible correction for nonresponse is 
incorporation of an additional equation for survey response into a two-equation system with 
the impact equation. The success of attempts to implement such corrections is data-dependent, 
and the differential response rates found do not seem quite large enough to warrant such 
uncertain measures. 

in. Im pact of Participation Versus Impact of A ssignment 

Because the target population for the JOBSTART Demonstration consisted of young 
people who had histories of dropping out of education programs, it was difficult to get those 
selected for the program to attend and to retain attendees for substantial periods of time. 
However, everyone assigned to experimental status was included when calculating average 

impacts of JOBSTART. Therefore, impacts do not measure the impacts of participation in 

3 

JOBSTART, but rather of assignment to the group eligible to receive JOBSTART services. 



^Some might suggest that nonparticipants be excluded from impact analyses. However, such exclusions 
would expose impacts to possible selection biases, undermining the control group's validity in measuring 
what would have happened without the program. When nonparticipants are excluded from the 
experimental group, average measured and unmeasured characteristics of experimentals may no longer be 
the same as average control group characteristics. See Cave, 1988. 
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Thus, impact estimates average net outcomes for all experimentals, including nonparticipants. 
Nonparticipation Vatere down" the program effect the experiment seeks to detect. Fortunately, 
only 106 of the 949 experimentals in the impact sample never participated in the program. 
Such low nonparticipation may have been due in part to successful negotiation with sites to 
place the point of random assignment after initial assessment but immediately before program 
services started. 

When substantial nonparticipation occurs during an experimental evaluation of a program, 
techniques are available for calculating impacts of participation as well as impacts of 
assignment. When the proportion of assignees to the program who are not counted as 
participants is an unbiased measure of the proportion of controls who would not have 
participated, when the program has no effect on nonparticipants, and when the sample is large 
enough, it is approximately valid to use the formula'* 

. . . Impact of assignment 
Impact of participation = . 

Fraction participating 

Using this formula necessitates validating all of the assumptions underlying it, and thus 
makes impact analysis more complicated than a simple comparison of average outcomes for 
those assigned to the experimental group and those assigned to the control group. The 
assumption of zero effects on nonparticipants is troublesome, because the process of recruiting 
experimentals, screening them, and contacting them when they do not appear may alter their 
behavior. Thus, in this report, impacts of assignment were reported instead of impacts of 
participation. 

As outlined above, impacts of assignment to JOBSTART were calculated by comparing 
average outcomes for all those assigned to the experimental group with average outcomes for 
all those assigned to the control group. In order to increase the statistical precLsion of the 
impact estimate, a variant of simple group averaging known as one-way linear analysis of 
covariance was used for the impact analysis in this report.^ As shown for the full sample of 
1,839 responders in Table B.4, in a multiple regression of outcome on covariates measured at 



^See Cave, 1988; Auspos, Cave, and Long, 1988, Appendix E; Bloom, 1984; and Parkas et al., 1984, 
p. 85. If such an adjustment factor were appropriate here, its value would be approximately the reciprocal 
of the rate of participation in JOBSTART or 1 / (1 - 106/949) = 1.126. 

'See Cave, 1987, and Ostle, 1975. 
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TABLE B.4 

ESTIMATED REGRESSION COEFFICIENTS FOR SELECTED OUTCOMES 



Dependent Variable 




Ever Received 


Received GEO or 








Any Education 


High School 








or 1 ra 1 n 1 ng 1 


uipioina oy tna ot 


Ever Employed, 


Total Earnings^ 


Regressor or Statistic 


Months 1-24 (%) 


Month 24 (%) 

* * 


Months 13-24 (%) 


Months 13-24 (i) 


Constant 


44.236*** 


16.443*** 


69.523*** 


4,098.812*** 




( 1 •LOO) 


(1.369} 


(1.420) 


(151.034) 


Experimental status 


48.418*** 


16.714*** 


2.473 


-204.732 




(1 .794 ; 


(1.938) 


(1.981) 


(210.715) 


Site 










ni icntuwii lOUi ToiCiJ 


1 7 nin*** 
1/ •UJU'""' 


-9.820* 


3.485 


917.304 




(4.788) 


(5.171) 


(5.286) 


(562.332) 




u .y/ J 


-19.982*** 


6.922 


1,659.399** 




(6.05Z) 


(6.536) 


(6.681) 


(710.750) 


D On \n 1 C ) 




-7.046 


2.701 


3,142.873*** 




(4.752) 


(5.132) 


(5.245) 


(558.026) 


LLi/oan Jose 


-20.218*** 


-5.824 


8.737* 


3,948.982*** 




(4.251) 


(4.591) 


(4.692) 


(499.199) 


Lnica^o ^.oniiions 


10 .o4j^ 


Oil OOIA^A^ 

-<4.383*** 


8.587 


1,812.704*** 




(5.608) 


•6.056) 


(6.190) 


(658.529) 


PnnnAlljit/ < Pi f' f cKiii^/«K ^ 


A Q1 Ti 
4 .OiU 


•0.474 


-0.795 


-443.437 




(4.612) 


(4.981) 


(5.091) 


(541.634) 


fRFf ^HartfnrH) 


? OA T 


1 C CI 74AiV 


2.164 


2,517.177*** 




(5.138) 


(5.549) 


(5.671) 


(603.386) 


nasi Ln 


7 9ni 


-28.80/'** 


-0.369 


2,238.462*** 


Skills Center 


(4.740) 


(5.119) 


(5.232) 


(556.584) 




'4 . //J 


-lo*lo4*** 


-3.365 


670.499 




(3.967) 


(4.284) 


(4.379) 


(465.856) 


Li Lcni.ro \Va} las^ 




2.323 


2.198 


1,177.852** 




(4.427) 


(4.781) 


(4.887) 


(519.907) 


LA Job Corps 


-3.823 


-19.036*** 


-2.649 


1,988.714*** 




(4.123) 


(4.453) 


(4.551) 


(484.163) 


Phoenix Job Corps 


-3.772 


-17.826*** 


-0.550 


674.547 




(4.422) 


(4.776) 


(4.881) 


(519.320) 


SER/Corpus Christi 










Age 20 or 21 


-0.586 


-1.782 


-2.868 


115.810 




(2.188) 


(2.363) 


(2.415) 


(256.968) 



(continued) 
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TABLE B.4 (continued) 



Dependent Variable 



Regressor or Statistic 
Male 

Ethnicity 
White, non-Hispanic 

Black, non-Hispanic 
Hispanic 

Other 



Left school in 
grade 11 or 12 



Ever Received 
Any Education 
or Training, 
Months 1-24 (%) 

-8.165*** 
(2.341) 



-3.244 
(3.633) 



Limited English 



No phone number on 
enrollment form 



Never married 



Male parent 



Female parent living 
with own child(ren) 



Lived with both parents 
at age 14 



Own AFDC case 
Household AFDC case 
Receiving Medicaid 
Receiving Food Stamps 



-1.887 
(2.671) 

4.504 
(5.798) 



-1.804 
(1.938) 

-5.127 
(5.000) 



-11.204"** 
(4.353) 

-4.409 
(3.381) 

-3.261 
(4.152) 



-7.048** 
(2.949) 



0.768 
(2.007) 

5.091* 
(3.096) 

4.717* 
(2.862) 

-1.968 
(2.697) 

-1.989 
(2.585) 



Received GEO or 
High School 
Diploma by End of 
Month 24 (%) 

-0.803 
(2.528) 



13.447*** 
(3.924) 



-1.529 
(2.885) 

■11.583* 
(6.262) 



6.357*** 
(2.093) 

9.664* 
(b.4O0) 



-10.430** 

(4.701) 

-0.809 
(3.652) 

-5.026 

(4.484) 



-1.715 
(3.185) 



0.732 
(2.167) 

-0.790 
(3.344) 

-3.336 
(3.091) 

-0.236 
(2.912) 

5.231* 
(2.792) 



Ever Employed, Total Earnings, 
Months 13-24 (%) Months 13-24 ($) 



17.626*** 
(2.584) 



14.017*** 
(4.010) 



6.708** 
(2.948) 

-1.077 
(6.400) 



5.989** 
(2.139) 

3.778 
(5.520) 



4.158 
(4.805) 

-6.274* 
(3.732) 

-3.077 
(4.583) 



-9.240*** 
(3.255) 



4.965** 
(2.215) 

-4.345 
(3.418) 

3.722 
(3.159) 

-10.455*** 

(2.977) 

^0.925 
(2.853) 



2,249.965*** 
(274.896) 



1,752.633*** 

(426.657) 



581.451* 
(313.667) 

964.634 
(680.916) 



819.836*** 
(227.585) 

545.046 
(587.227) 



-290.917 
(511.200) 

-982.086** 
(397.063) 

629.133 
(487.604) 



-320.469 
(346.287) 



680.035*** 
(235.652) 

-244.139 
(363.609) 

-108.485 
(336.085) 

-906.072*** 
(316.671) 

38.620 
(303.572) 



(continued) 
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TABLE B.4 (continued) 

Dependent Variable 



Ever Received Received GED or 

Any Education High School 

D r. . °'" ^"""^"^"a' Diploma by End of Ever Employed, Total Earninas 

Regressor or Statistic Months 1-24 (V) Month 24 (%) Months 12-2^ (H) Months 13 24 Ji) 



Employed within past year 
Arrested since age 16 
Convicted since age 16 
Number of 



-2.521 
(1.943) 

-2.371 
(3.187) 

2.605 
(4.835) 



0.081 
(2.099) 

-2.763 
(3.442) 

-0.232 
(5.222) 



5.370** 


1,001.186*** 


(2.145) 


(228.190) 


-4.801 


-384.891 


(3.518) 


(374.237) 


-0.793 


-377.090 


(5.337) 


(567.844) 



observations 


1,839 


1,839 


1,839 


1,839 


Number of 
experimental s 


949 


949 


949 


949 


Number of controls 


890 


890 


890 


890 


Degrees of freedom 
for error 


1,806 


1,806 


1,806 


1,806 


trror mean square 


1.465.154 


1,708.907 


1,785.134 


20,205,656.707 


R square 


0.325 


0.107 


0.152 


0.203 


Mean of dependent 
variable 


69.222 


25.068 


70.799 


3,993.162 


F-statistic 


27.13 


6.73 


10.12 


14.36 


P-value of F-statistic 


0.000 


0.000 


0.000 


0.000 



SOURCE: MDRC calculations from J08START enrollment fom. MIS, and survey data. 
presellJed'in TablesT! Ts l7Z/Tr':" ''^'''V"''' ''^^ '^''^ "-"pond to impact estimates 

its =H."ct.r,stK o„ the righl-h.nd side .f e.ch equation ,.s .e«.red « . de.inio. from 

a. indicated ,1 ";'re^^^••^T;:rt "'t 
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the time of enrollment and on a dummy variable for research status, the coefficient of the 
dummy variable is the impact. This coefficient may be interpreted as the difference between 
the adjusted mean outcome for those assigned to the experimental group and the adjusted 
mean outcome for those assigned to the control group. Adjustment removes the effect of 
slight differences at the time of enrollment in characteristics related to the outcome, and yields 
a purer measure of the effect of research status alone. 

Some of the subgroup results presented in the report were based on slightly more 
complex regression equations, which include terms for interactions between experimental status 
and subgroup characteristics. Such "two-way ANCOVA" impacts may differ to some extent 
from "split-file" impacts estimated by eliminating other subgroups from "one-way ANCOVA" 
analyses for Table B.4. However, calculating two-way ANCOVA impacts permits determming 
the statistical significance of impact differences, and is less burdensome computationally. 

IV. The Tntex-nal V fl'^'*y rnm parisnns Among Subgroups and ly pes pISitcs 

Youths in the impact sample can be grouped based on their individual, pre-random 
assignment characteristics or op the characteristics of the sites at which they applied for 
JOBSTART. Because such subgroup comparisons are a centra! part of the analysis presented 
in this report, it is important to discuss briefly the complications in drawing ronclusions from 
any observed differences. Crucial comparisons of this type are between men and women, and 
between types of sites. To summarize, impacts can be compared across subgroups defined by 
individual or site characteristics, but more caution is advised in interpreting such results than 
in interpreting the full sample or within-site impacts just described. This is especially true for 

comparisons of site types. 

The basic reason is that, since sample members were not assigned randomly to these 
subgroups or types of sites, it may be impossible to isolate the difference in impact attributable 
to the single characteristic used to designate the groups. For example, using an example of 
a group "defined" by an individual characteristic, if women have bigger impacts than men, it 
may not be because they are women; the impact difference might really be because they had 
less prior work experience so controls were less likely to be working in the follow-up period. 
Further, asing site or site groupings for a subgroup impact comparison is fundamentally 
different from using an individual characteristic such as gender. Many things about sites differ 
(such as labor market, participant characteristics and interests, and program characteristics), and 
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there is a real danger that impact differences for site groupings may be misinterpreted as 
measuring the relative efficiency of a single feature of a site's program, such as its curricula, 
facilities, or program structure (that is, brokered versus in-house services or concurrent versus 
sequential education and training) rather than other factors that could be drivkig inter-site 
variation. 

The internal validity of impact comparisons by individual characteristics is difficult to test. 
However, there is a simple test for internal validity of impact comparisons by program features. 
If groups of individuals randomly assigned at two locations really differed only in the features 
of the programs experimentals could attend, then the post-random assignment experience of 
controls at the two locations should be identical. This rarely happens; more typically, the 
experience of controls varies between sites, just as that of experimentals does.^ 

This problem can also affect impact comparisons for subgroups defined by characteristics 
of individuals as well as subgroup impacts by site or site grouping, although ordinarily the 
concerns about misinterpretation are less severe. For example, if virtually all the Hispanics 
recruited into a demonstration were concentrated in one or two sites, then "Hispanic" impacts 
really were impacts for "those sites." However, normally (and in the JOBSTART 
Demonstration, as discussed below) most measured characteristics of individuals are distributed 
fairly evenly across locatioas.^ There will have been younger and older sample members, 
people reading at higher and lower levels, and parents and childless youths in samples recruited 
in all JOBSTART sites. Moreover, relevant unmeasured characteristics of individuals such as 
strength of motivation to attend a program and desire for a GED are likely to have been 
distributed fairly evenly among younger versus older sample members, those with higher versus 
lower reading levels, those who were parents versus those who were childless, and other 
subgroups defined by the observed characteristics of individuals.^ If the impact on educational 
attainment, for example, was higher among low-reading-level than among high-reading-level 
sample members, it is reasonable to interpret this as evidence that the programs were more 



usual situation is what is known in the evaluation literature as "ecological correlation bias." 

''As discussed in the following section, several ethnic groups were highly concentrated in a few 
JOBSTART sites, however. 

^o lessen this problem, impact estimates presented in this report always used site dummies &s 
covariates when calculating impacts by individual characteristics; these dummies can correct for small 
differences between subgroups defined by individual characteristics in unmeasured characteristics associated 
with site. 
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effective with the former rather than concluding that higher educational attainment occurred 
because the average sample member who had a low reading level was better motivated or 
wanted a GED more than did the average sample member who had a higher reading level. 

In contrast, when a sample is split by site or site group, unmeasured characteristics will 
be distributed unevenly across groups. For example, sites that were known for providing 
education services and that offered a sequence of basic skills instruction followed by 
occupational training at another agency (sequential/brokered sites) may have been more likely 
to recruit clients motivated to get a GED than did concurrent sites with a reputation for 
training, whose typical client may have wanted to learn occupational skills. Thus, a finding that 
JOBST ART'S impact on GED attainment was less at concurrent sites than at 
sequential/brokered sites does not necessarily mean that someone with average motivation and 
desire for a GED has a better chance of getting a GED in a sequential program. Such a 
finding could mean that those recruited in sequential locations had on average very different 
levels of desire for skills training relative to GED preparation from those recruited at 
concurrent locations. Even if the usual statistical adjustment methods are employed in 
calculating impacts, little can be done about this problem, since motivation to participate in 
particular components was not measured.^ 



'Subgroup impact equations for groupings by site type cannot use individual site dummies to correct 
for small unobserved differences in groups in the same way that equations for groups defined by individual 
characteristics can: Individual site dummies would be highly correlated with the site groupings. To measure 
which deliveiy system is better for those with average levels of motivation, desire for GEDs, and other 
unobserved characteristics, the best approach is to randomly assign people to each dsliveiy system in each 
location after carrying out a common recruitment effort at that location. In that way, unmeasured 
characteristics would be the same for each deliveiy system, because each deliveiy system would be fairly 
represented in each location and in each recruitment effort. Other, nonexperimental approaches to this 
problem will be attempted in future stages of the JOBSTART research when longer follow-up is available. 
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APPENDIX C 
SUPPLEMENTAL TABLES TO OfiAPTER 2 



TABLE C.l 



CHARACTERISTICS IN THE YEAR BEFORE RANDOM ASSIGNMENT, 
BY EMPLOYMENT STATUS 



Employed Within the Year 



Characteristic and Subgroups 


Sample 
Size 


No 


Yes 


Both 
Groups 




Gender 












Wom^n 


968 


62.9% 


43.4% 


52.6%*** 


0.000 


Men 


871 


37.1 


56.6 


47.4 




Ethnici ty 












White, non-Hispanic 


155 


7.1 


9.6 


8.4** 


0.015 


Black, non-Hispanic 


840 


46.4 


45.0 


45.7 




Hispanic 


783 


42.0 


43.1 


42.6 




Other 


61 


4.5 


2.3 


3.3 




Ethnicity, by gtnder 
























White, non-Hiipanic 


84 


4.5 


4.6 


4.6*** 


0.000 


Black, non-His|.anic 


445 


23.3 


20.5 


24.2 




Hispanic 


413 


28.3 


17.2 


22.5 




Other 


26 


1.8 


1.0 


1.4 




Men 












White, non-Hispanic 


71 


2.6 


5.0 


3.9 




Black, non-Hispanic 


395 


18.2 


24.5 


21.5 




Hispanic 


370 


13.7 


25.9 


20.1 




Other 


35 


c . u 


' 1 9 


1 0 




Parental status 












Women living with 












own child(ren) 












No 


484 


28.2 


24.7 


26.3*** 


0.000 


Yes 


484 


34.7 


18.8 


26.3 




Men who have own child(ren) 












No 


7t5 


33.6 


48.8 


41.6 




Yes 


10b 


3.6 


7.7 


5.8 




Prior employment, by gender 












Women employed within 












past year 












No 


547 


62.9 


0.0 


29.7*** 


0.000 


Yes 


421 


0.0 


13.4 


22.9 




Men employed within 












past year 












No 


323 


37 1 


0.0 


17.6 




Yes 


548 


O.vl 


56.6 


29.8 




Sample size 


1,839 


870 


969 
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TABLE C.l (continued) 



Sample 

Characteristic and Subgroups Size 



Employed Within the Year 
Before Random Assignment 



No 



Yes 



Both 
Groups 



Left school in grade 11 or 12 
No 1,078 
Yes 761 



59.1V 
40.9 



58.2% 
41.8 



58.6% 
41.4 



0.703 



Received occupational 
training within past year 

No 

Yes 



1.529 
310 



87.0 
13.0 



79.7 
20.3 



83.1*** 
16.9 



0.000 



Age 
16-19 
20 or 21 



1,359 
480 



74.5 
25.5 



73.4 
26.6 



73.9 
26.1 



0.589 



Marital status 
Ever married 
Never married 



174 
1.665 



9.2 
90.8 



9.7 
90.3 



9.5 
90.5 



0.712 



Living in own household or 
with boy/girlfriend 

No 

Yes 



1,500 
339 



79.2 
20.8 



83.7 
16.3 



81.6** 
18.4 



0.013 



Own AFDC case or receiving 
General Assistance 

No 

Yes 



1.344 
495 



66.0 
34.0 



79.5 
20.5 



73.1*** 
26.9 



0.000 



Own AFDC case 
No 
Yes 



1.446 
393 



70.6 
29.4 



85.9 
14.1 



78.6*** 
21.4 



0.000 



Receiving Food Stamps 
No 
Yes 



1.143 
696 



58.0 
42.0 



65.8 
34.2 



62.2*** 
37.8 



0.001 



Arrested since age 16 
No 
Yes 



1.567 
272 



88.5 
11.5 



82.2 
17.8 



85.2*** 
14.8 



0.000 



Lived with both parents at 
age 14 

No 

Yes 



1.198 
641 



68.5 
31.5 



62.1 
37.9 



65.1*** 
34.9 



0.004 



Sample size 1,839 870 969 
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-218- 

r) : 



TABLE C.l (continued) 



Characteristic and Subgroups 



Sample 
Size 



Employed Within the Year 
Before Random Assignment 



No 



Yes 



Both 
Groups 



Site 

Concurrent 

Atlanta Job Corps 61 

CET/San Jose 152 

Chicago Conmons 74 

Connelley (Pittsburgh) 184 

East U Skills Center 100 

EGOS (Denver) 183 

Phoenix Job Corps 130 

SER/Corpus Christi 236 

Sequential/in-house 

El Centro (Dallas) 155 

LA Job Corps 218 

Sequential /brokered 

Allent-^wn (Buffalo) 140 

BSA (NYC) 119 

CREC (Hartford) 87 



2.6% 

6.4 

4.6 

6.1 

6.3 

8.4 

8.3 

9.1 

8.0 

18.9 

9.3 
8.9 
3.1 



3.9% 

9.9 

3.5 

13.5 
4.6 

11.4 
6.0 

16.2 

8.8 
5.6 

6.1 
4.3 
6.2 



3.3%*** 

8.3 

4.0 
10.0 

5.4 
10.0 

7.1 
12.8 

8.4 
11.9 

7.6 
6.5 
4.7 



0.000 



Sample size 



1,839 



870 



969 



SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all 1,839 sample members for whom 
there were 24 months of follow-up survey data. Sample sizes reported may fall short of this 
number because of items missing from some sample members' questionnaires. 

Distributions may not total 100.0 percent because of rounding. 

*The column labeled "p" is the statistical significance level of the 
difference in distributions of characteristics between groups: that is, p is the probability 
that observed proportions in each subgroup differ by employment status only because of 
random error. A Pearson chi-square statistic was used to test the hypothesis of equal 
distributions. Statistical significance levels are indicated as *** = 1 percent; ** = 5 
percent; * » 10 percent. 
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TABLE C.2 



CHARACTERISTICS IN THE YEAR BEFORE RANDOM ASSIGNMENT, 
BY GENDER AND EMPLOYMENT STATUS 



I 

NJ 
NJ 
O 
I 



ERIC 



Characteristic and Si'jgroups 



Sample 
Size 



Women 

Not 

Employed Employed 



Hen 



Not All 
Empl oyed Empl oyed Categori es 



Ethnici ty 
















White, non-Hispanic 


155 


7.1% 


10.7% 




O . O't 


Q ^ n n n 


U • UU 1 


Black, non-Hispanic 


840 


45.0 


47 3 


dfl Q 
HO . J 




AC 7 


Hispanic 


783 


45.0 


39.7 


36.8 


45.8 


42.6 




Other 


61 


2.9 


2.4 


7.1 


2.2 


3.3 




Ethnici tv. bv aender 
















Women 
















White, non-Hispanic 


84 


7.1 


10.7 


0.0 


0 n 


H . 0 


n nnn 


Black, non-Hispanic 


445 


45.0 


17. 3 


0 0 


n n 


OA 0 




Hispanic 


413 


45.0 


39.7 


0 Q 


n n 






Other 


26 


2.9 


2.4 


0.0 


n n 


1 A 




Men 














White, non-Hispanic 


71 


0.0 


0.0 


7.1 


8.d 


3.9 




Black, non-Hispanic 


395 


0.0 


0.0 




H J . C 






Hi spanic 


370 


0.0 


0.0 


36.8 


45.8 






Other 


35 


0.0 


0.0 


7.1 


2.2 


1.9 




Parental status 
















Women living with own 
















chi Id(ren) 
















No 


484 


44.8 


56.8 


0.0 


0.0 


26.3*** 


0.000 


Yes 


484 


55.2 


43.2 


0.0 


0.0 


26.3 




Men who have own child(ren) 
















No 


765 


0.0 


0.0 


90.4 


86.3 


41.6 




Yes 


106 


0.0 


0.0 


9.6 


13.7 


5.8 




Left school in grade 11 or 12 
















No 


1,078 


58.9 


61.0 


59.4 


56.0 


58.6 


0.449 


Yes 


761 


41.1 


39.0 


40.6 


44.0 


41.4 




Sample size 


1,839 


547 


421 


323 


548 
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TABLE C.2 (continued) 



Sample 

Characteristic and Subgroups Size 



. Women 

Not 

Employed Employed 



Men 

Not All 
Employed Employed Categories 



I 



Received occupational training 

within past year 

>io 1,529 
Ves 310 



Age 
16-19 
20 or 21 

Marital status 
Ever married 
Never married 

Living in own household or 
with boy/girlfriend 

No 

Yes 

Own AFDC case or receiving 
General Assistance 

No 

Yes 

Own AFDC case 
No 
Yes 

Receiving Food Stamps 
No 
Yes 

Arrested since age 16 
No 

Yes 



1,359 
480 



174 
1»665 



1,500 
339 



1,344 
495 



1,446 
393 



1,143 
696 



1,567 

272 



89.4V 
10.6 



72.9 
27.1 



13.9 
86.1 



68.9 
31.1 



55.0 
45.0 



58.7 
41.3 



51.6 
48.4 



95.6 
4.4 



84.1% 
15.9 



73.9 
26.1 



11.4 
88.6 



74.3 
25.7 



68.2 
31.8 



72.9 
27.1 



58.0 
42.0 



94.1 
5.9 



83.0V 
17.0 



77.1 
22.9 



1.2 
98.8 



96.6 
3.4 



84.5 
15.5 



90.7 
9.3 



69.0 
31.0 



76.5 
23.5 



76.3V 
23.7 



73.0 
27.0 



8.4 
91.6 



90.9 
9.1 



88.1 
11.9 



95.8 
4.2 



71.9 
28.1 



73.2 

26.8 



83.1V*** 0.000 
16.9 



73.9 
26.1 



9.5*** 
90.5 



62.2*** 
37.8 



85.2*** 
14.8 



0.532 



0.000 



81.6*** 0.000 
18.4 



73.1*** 0.000 
26.9 



78.6*** 0.000 
21.4 



0.000 



0.000 



Sample r, izo 



1 ,839 



547 



421 



323 



548 
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TABLE C.2 (continued) 



Characteristic and Subgroups 



Women 

Sample Not 
Size Employed Employed 



Men 



Not 



All 



Empl oyed Employed Categori es p^ 



Lived with both parents at 



age 14 














No 


1,198 


70.7% 


67 2k 

Ml • 


fid lie 


Do . 


65 . 1^*** 


Yes 


641 


29.3 






HI ,0 


34 . V 


Site 














Concurrent 














Atl anta Job Corps 


61 


3 3 


a 1 




3.5 


3 • 3*** 


CET/San Jose 


152 


6.2 




D.O 


in n 


0 • 3 


Chicago Commons 


74 


3.7 




ft 9 


A n 


4.0 


Connelley (Pittsburgh) 


184 


6.2 


15.2 


5.9 


12.2 


10.0 


East LA Skills Center 


100 


5.1 


3.8 


8.4 


5.3 


5.4 


EGOS (Denver) 


183 


10.6 


14.3 


4.6 


9.1 


10 n 

A V/ • V/ 


Phoenix Job Corps 


130 


8.6 


4.8 


7./ 


6.9 


7.1 


SER/Corpus Christ i 


236 


9.3 


10.0 


8.7' 


21.0 


12.8 


Sequential/in-house 














El Centro (Dallas) 


155 


7.9 


9.7 


8.4 


8.0 


8.4 


LA Job Corps 


218 


18.8 


6.4 


18.9 


4.9 


11.9 


Sequential /brokered 














Allentown (Buffalo) 


140 


10.1 


6.4 


8.0 


5.8 


7.6 


BSA (NYC) 


119 


6.4 


4.3 


13.0 


4.4 


6.5 


CREC (Hartford) 


87 


3.8 


8.3 


1.9 


4.6 


4.7 


Sample size 


1,839 


547 


421 


323 


548 





0.000 



SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all 1,839 sample members for whom there were 24 months 
of follow-up survey data. Sample sizes reported may fall short of this number because of items missing from some 
sample members* questionnaires. 

Distributions may not total 100.0 percent because of rounding. 

*The column labeled "p" is the statistical significance level of differences among groups in 
distributions of characteristics: that is, p is the probability that observed proportions in each subgroup differ 
from one column to another only because of random error. A Pearson chi-square statistic was used to test the 
hypothesis of equal distributions. Statistical significance levels are indicated as = 1 percent; = 5 
percent; * = 10 percent. 



TABLE C.3 



CHARACTERISTICS AT THE 


TIME OF RANDOM 


ASSIGNMENT, 


BY AGE 






Sampl e 




Age 20 


Both 




Char«cter1stic and Subgroups 


Size 


Age 16-19 


or 21 


Groups 




Gender 












Women 


968 


52.2% 


53.7% 


52.6% 


0.570 


Men 


871 


47. P 


46.2 


47.4 




Ethnicity 












White, non-Hispanic 


155 


9.8 


4.6 


8.4*** 


0.000 


Black, nor-Hispanic 


840 


It . 0 




AC 7 




Hispanic 


783 


44.3 


37.7 


42.6 




Other 


61 


3.3 


3.3 


3.3 




Ethnicity, by gender 












Women 










• 


White, non-Hispanic 


84 


5.1 


3.1 




0.000 


Black, non-Hispanic 


445 


21.7 


31.2 


24.2 




Hispanic 


413 


24.0 


18.1 


22.5 




Other 


26 


1.5 


1.2 


1.4 




Men 












White, non-Hispanic 


71 


4.7 


1.5 


3.9 




Black, non-Hispanic 


395 


on o 


c6 . 1 






Hispanic 


370 


20.3 


19.6 


20.1 




Other 


35 


1.8 


2.1 


1.9 




Parental status 












Women living with own 












child(ren) 












No 


484 


30.3 


15.0 


26.3*** 


0.000 


Yes 


484 


21.9 


38.7 


26.3 




Men who have own child{ren) 












No 


765 


43.9 


35.2 


41.6 




Yes 


106 


3.9 


11.0 


5.8 




Employed within past year 












No 


870 


47.7 


46.2 


47.3 


0.589 


Yes 


969 


52.3 


53.7 


52.7 




Prior employment, by gender 












Women employed within 












past year 












No 


547 


29.4 


30.8 


29.7 


0.532 


Yes 


421 


22.9 


22.9 


22.9 




Men employed within pas<" year 












No 


323 


18.3 


15.4 


17.6 




Yes 


548 




in fi 
ju .o 






Sample size 1 


,839 


1,359 


480 
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TABLE C.3 (continued) 



Sample Age 20 Both 

Characteristic and Subgroups Size Age 16-19 or 21 Groups 



Left school in grade 11 or 12 
No 1,078 
Yes 761 



61.4^ 
38.6 



50.6% 
49.4 



58.6%*** 
41.4 



0.000 



Receivsd occupational 
training within past year 

No 

Yes 



1.529 
310 



Q3.4 
16.6 



82.5 
17.5 



83.1 
16.9 



0.662 



Marital status 
Ever married 
Never married 



174 
1.665 



7.4 
92.6 



15.2 
84.8 



9 5*** 
90.5 



0.000 



Living in own household or 
with boy/girlfriend 

No 

Yes 



1.500 
339 



86.3 
13.7 



68.1 
31.9 



81.6*** 
18.4 



0.000 



Own AFDC case or receiving 
General Assistance 

No 

Yes 



1,344 
495 



78.7 
21,3 



57.3 
42,7 



73.1*** 
26.9 



0.000 



Own AFDC case 
No 
Yes 



1.446 

393 



83.0 
17.0 



66.2 
33.7 



78.6*** 
21.4 



0.000 



Receiving Food Stamps 
No 
Yes 



1,143 
696 



65.0 
35.0 



54.0 
46.0 



62.2*** 
37.8 



0.000 



Arrested since age 16 
No 
Yes 



1,567 

272 



85.2 
14.8 



85.2 
14.8 



85.2 
14.8 



0.999 



Lived with both parents at 
age 14 

No 

Yes 



1.198 
641 



65.7 
34.3 



63.5 
36.5 



65.1 
34.9 



0.392 



Sample size 1.839 1.359 480 



(continued) 
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TABLE C.3 (continued) 





oanip 1 c 




Age c\j 


Both 




d 1 Zc 


Ana 1 ^ 10 


or 21 


Groups 


f A 










Loncurrcn b 










n\ \ an WO UOD Lorpa 


^ 1 


J • 3^ 


2. 9^ 


3.3%*** 






o • o 


t . 1 


Q 1 

0.3 




74 


9 A 
c • D 


Q 1 
O • 1 


A n 
4.U 


Ponnpl 1 PV f Pi 1 1 ^ hij rnh ^ 




7 A 


17 7 
1 / • J 


lU .U 


LGdv US jM 1 i9 Lenter 


inn 


C 7 


A C 






1 

loo 


in "J 




10.0 


rriucniA uuu ^urpa 


1 ^n 


O • c 


J. / 


7 . 1 


^PD/rrkv*niic Phv*i c f i 

dLr\/Lorpud LnriSti 






14.D 


12.8 


Sequential /in-house 










El Centro (Dallas) 


155 


9.6 


5.2 


6.4 


LA Job CorDS 


218 




in 9 


11 Q 

11.7 


Sequential /brokered 










Allentown (Buffalo) 


140 


7.6 


7.7 


7.6 


BSA (NYC) 


119 


6.5 


6.5 


6.5 


CREC (Hartford) 


87 


5.2 


3.5 


4.7 


Sample size 


1,839 


1,359 


480 





SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all 1,839 sample members 
for whom there were 24 months of follow-up survey data. Sample sizes reported may fall 
short of this number because of items missing from some sam;jle members* questionnaires. 

Distributions may not total 100.0 percent because of rouiiding. 

*The column labeled "p" is the statistical significance level of the 
difference in distributions of characteristics between groups: that is, p is the probability 
that observed proportions in each subgroup are different only because of random error. A 
Pearson chi-square statistic was used to test the hypothesis of equal distributions. 
Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 
percent. 
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APPENDIX D 
SUPPLEMENTAL TABLE TO CHAPTER 3 



TABLE D.l 



RATINGS AND SPECIAL FEATURES OF THE IMPLEMENTATION 
OF JOBSTART COMPONENTS, BV SITE 



Site 



Concurrent 
Atlanta Job Corps 



I 

I 



CET/San Jose 



Ratings and Special Features of Components 



Education 



Rating: Medium 
•self-paced. 

Individualized 

Instruction supplemented 

by group instruction 
•some computer-assisted 

instruction 
•unclear objectives 
•frequent staff turnover 

Rating: None 

•a rating of this 
component is 
Inappropriate because 
education and training 
were more integrated 
than in other sites, 
although there was 
separate GEO preparation 



Training 



Rating: Medium 

•good mix of on^ and off- 
site training 

•lacked state-of-the-art 
equipmeint 

•some employer and union 
presence 



Rating: High 

•open entry and exit 

•sel f-paced, 

individualized 

instruction 
•on-site training 
•concurrent education and 

training 

•instructors hired from 
industries about which 
they teach 

•participants had 
opportunity to observe 
different training 
classes before making a 
selection 



Support Services 



Rating: High 

• incentives for 
participation 

• on-site child care 

• strong counsel ing 

• heal th services 



Rating: Medium 

• on-site child care; 
not subsidized, but 
sliding scale 

• little individualized 
counseling; most 
provided by instructors 

•no incentive payments 



Job Placement 



Rating: Low 
•few linkages to 
employers 

•little special attention 
devoted to JOBSTART 
youths 



Rating: High 
•good relations with 
employers 

•job placement a high 
priority 
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TABLE D.l (continued) 



CI f A 




Ratinqs and Special Features 


of Components 




Educati on 


Training 


Support Services 


Job Placement 


Chicago Coimons 


Rating: Medium 


Rating: High 


Rating: Hedlum 


Rating: High 




•rigorous curricula 


' training provided in 


•relatively small case- 


•staff person fully dedi- 




coordinated with 


word processing and in 4 


loadsr but counselors 


cated to job development 




vocational requircmc/its> 


semiskilled industrial 


carried full responsi- 


•extensive ties to 




allowing for immediate 


trades 


bility for recruitment 


employer com.iunity 




appi ication and 


•rigorous curricula that 


during periods 


•all placements training- 




reinforcement of 


incorporated input from 


•referrals for child care 


related 




learning 


employers 


•JTPA needs-based pay- 


•given demanding nature 




•special tutorial work- 


•hands-on instruction 


ments for transportation 


of potential placements, 




shops for JOBSTART 


almost daily 


and lunchi but checks 


youths needed more 




parti c \ pants 


• certi f i ed i nstruc trrs 


wcrc irci|ucnwijr iqwc 


exposure to employment 




•no computer-assisted 


recruited from work 


• adult-oriented 


environment prior to job 




instruction 


settings students were 


environment 


interviews (began visits 




•because of vocational 


preparing to enter 




to workplaces during 




focus» GEO preparation 


• state-of-the-art 




later months of the 




almost precluded 


equipment 




demonstration's 






• instructors adult- 




operating period) 






oriented» not always 










sensitive to youths' 










developmental needs 






Conncllcy (Pittsburgh) 


Rating: High 


Rating: High 


Rating: High 


Rating: Medium 




•clear objectives 


•multiple training areas 


• referrals to other 


•informali individualized 




•excel lent i nf rastructure 


•combined basic skills 


aaenc i es 


J yfU aCa 1 V M 




•computer-assisted 


and vocational training 


•on-site child care 


•too few job developers 




i nstruction 


•interactive learning 


•needs-based payments and 


to provide focused 




•combined occupational 


•strong employer presence 


incentives 


assistance to JOBSTART 




training with basic 




•special JOBSTART 


youths 




education and GEO 




counselors 


•poor attention to world 




preparation in some 




•some clothing assistance 


of work/job-readiness 




cases 




•special retention 






'tutorial assistance 




activi tie*; 






•multimedia approach 
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TABLE D.l (continued) 



Ratings and Special Features of Cotnponents 



Site 



Education 



Training 



Support Services 



Job Placement 



East LA 
Skills Center 



EGOS (Denver) 



Rating: Medium 

•clear objectives 

•licensed instructors 

• individualized 
instruction supplemented 
by group instruction 

•some effort to combine 
occupational training 
and GEO preparation 

•some use of the computer 
lab 



Rating: Medium 

•open entry and exit 

•self-pacedp 
individual ized 
instruction 

•JOBSTART-only classes 

•on-site classes 

•on-site GEO testing with 
almost immediate 
reporting of results 

•computer-assisted 
instruction available, 
but not widely used 



Rating: Medium 

•good mix of training 

areas on-site 
•some staff turnover and 

some uninspired 

instructors 

•combined occupational 

training with basic 

education and GEO 

preparation 
•good relations with 

employers 

•crowded classrooms made 
it difficult for some 
youths to get the 
attention of instructors 

Rating: Medium 
•not always open entry 
and exit 

•several training 
programs required more 
time than participants 
had avai lable 

• training staff used to 
teaching adults, not 
always pleased to have 
JOBSTART participants in 
class 



Rating: Medium 
•counseling provided 
•no incentives for 

participation 
•limited transportation, 

mcal» 0.' financial 

assist/mce 



Rating: Medium 
•child care payments 

• free bus passes 

• trained, caseload- 
carrying counselors for 
participants 

•referrals for other 
services 

• good attendance 
enforcement 

•no incentives to 
participate except 
occasional parties 



Rating: Low 
•some supervised, 

individualized job 

search 

•one placement staff - 
person 

•little attention to 
JOBSTART youths 



Rating: Low 

•no specific job 

development staff 
•few successful 

completions of training 

and transitions to 

employment 
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TABLE D^l (continued) 



Ratings and Special Features of Components 



Site 



Education 



Training 



Support Services 



Job Placement 



Phoenix Job Corps 



I 
I 



SER/Corpus Christi 



Rating: Medium 
•Individualizedt 

competency-based 

instruction 
•computer-assisted 

instruction 
•open entry and exit 
•little opportunity for 

group instruction/ 

interaction 



Rating: High 
is^ell -articulated curri- 
culum and weekly plans 

•used computer-assisted 
instruction to supple- 
ment GEO preparation 

•well-articulated atten- 
dance and punctuality 
standards 

•some effort to integrate 
vocati onal instruct! on 
with GEO preparation 

•clear performance out- 
comes were attained 



Rating: High 
'individualized, 

competency-based 

instruction 
•open entry and exit 
•wide range of courses 

available 

•certified instructors 
'little opportunity for 

group instruction/ 

interaction 



Rating: Low 

•on-site training, but 

limited to clerical and 

automotive skills 
•lacked stftte-of-thc-art 

equipment 

•difficulty attracting 
and retaining qualified 
staff 

'limited linkages with 
the private sector 

•maintained fairly high 
student morale and 
attendance despi te 
shortcoming!; 



Rating: High 

•financial incentives for 
attendance and 
performance 

•structured procedures 
for reviewing partici- 
pants' performance 

•youth-oriented 
environment 

• relatively large 
caseloads 

•referrals for child 
carev but limited 
community rcsou' ,es 

Rating: High 

•needs-based payments for 
transportation and meals 

•referrals for child care 
to other SER locations 
or other providers 

• incentive payments 
•strong counseling 



Rating: Low 

•several staffpersons 

fully dedicated to job 

development 

*job development services 
offered to non- 
completers in reasonably 
good standing at program 
departure 

•many placements not 
training-related 



Rating: Low 

•responsibil i ty for 
placement rested with 
Texas Employment 
Cownission, which did 
not consider placement 
of JOBSTART youths a 
priority 
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TABLE 0,1 (continued) 



Ratings and Special Features of Components 



Site 



Education 



Training 



Support Services 



Job Placement 



Sequent ial/in-house 
El Centro (Dallas) 



LA Job Corps 



Rating: High 

•clear objectives 

•well -designed curriculum 

•good mix of individ- 
ualized and group 
instruction 

•efforts to integrate 
vocational curriculum 

•tutorial assistance 

•reading and learning lab 
for those needing extra 
help 

•flexible program 
' integration of field 
trips and current events 
with GEO preparation 
•multimedia approach 
•licensed instructors 
•community college 
infrastructure 

Rating: Medium 
•clear objectives 
•stable staff 
•multimedia approach 
• interactive learning 



Rating: Medium 

•clear objectives 

•7 on-site training areas 

•certified instructors 

with employer contacts 
• state-of-the-art 

equipment in cable TV 

and clerical training; 

fairly current equipment 

in other areas 
'interactive learning 
•good curriculum 



Rating: Medium 
•good mix of training 
areas 

•stable staff 
•good employer and union 
presence 

•state-of-the-art equip- 
ment in most areas 

•on- and off-site 
1 inkages 



Rating: Hediwn 
•needs-based payments 
•bus passes 

• incentive payments 
•some emergency rent 

assistance 

• good counsel ing 



Rating: High 
• incenti ves 
• strong counsel ing 
•health and other 
services 



Rating: Medium 

•job placement staff 

responsible for all El 

Centro students 
•limited attention to 

JOBSTART youths 



Rating: Medium 
•good relations with 
employers 

-some supervised job 
search 

•Ignited attention to 
JOBSTART youths 
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TABLE 0.1 (continued) 



Ratings and Special Features of Components 



Site 



Education 



Training 



Support Services 



Job Placement 



Sequen tial/b roke red 
Allentown (Buffalo) 



BSA (NYC) 



Rating: High 

*computer*ass{ sted 
instruction 

•individualized 
instruction supplemented 
by group instruction 

•interactive learning 

•GEO preparation focus 



Rating: Medium 

•good mix of academic, 

job-readiness, and life 

skills activities 
-computer-assisted 

i nstruction 
•fairly limited GEO 

preparation 



Rating: Low 

•difficulty in transition 
of youths to JTPA and 
proprietary schools 

•no on-site training 



Rating: Low 
•limited referrals 
•lack of success linking 
up with JTPA 



Rating: High 

• counsel ing 
•needs-based payments 
•meals and transportation 
•referrals for medical, 

dental , and other 
servi ces 
•no incentive payments 
for parti cipation 

Rating: Medium 

• strong counseling 

• strong relationship 
between staff and 
participants 

•some needs-based pay- 
ments for a portion of 
the participants 

• referrals for other 
servi ces 

•incentive payments for 
part of demonstration 



Rating: Low 

•limited direct placement 

assistance provided to 

JOBSTART youths 
•reliance on placement 

staff at training 

agencies 



Rating: Low 

• few successful 
completions of training 
and transitions to 
employment 

•limited involvement with 
employers 
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TABLE D,l (continued) 



Ratings and Special Features of Components 



Site 



Education 



I 

M 

Ln 
I 



CREC (Hartfora) 



Rating: Low 
'computer-assisted 
instruction 

•limited GEO preparation 
•underutilized computer 

learning center 
•limited use of group 

instruction 
•attrition and low 

attendance 



Training 



Rating: Low 
•limited access to 
training 

•poor linkages to 
eniployers 

•few transitions to 
training 



Support Services 



Rating: Medium 

* counsel ing 

*bus passes 

' referrals for other 

services 
•no needs-based payments 
•no incentive payments 

for participation 



Job Placement 



SOURCE: 



Observations of MDRC operations and research staff during the period of program operation in each site. 



Rating: Low 

•limited involvement with 
training or employer 
community 

■limited attention to 

JU3START youths 
• informal job search 

assistance 
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APPENDIX E 
COST OF THF TOBSTART PROGRAM 



I. (;gine ral Approach 

This appendix describes the data sources and methodology used to estimate the cost of the 
JOBSTART program in each of the 13 demonstration sites. It also discusses the factors 
contributing to the wide variation in costs across sites and examines the relative influence of 
different JOBSTART components on overall program costs. 

The central objective of the anaiy-sis was to identify the market value of all resources used 
in providing JOBSTART services. It therefore counted as program costs not only the 
expenditures made by the agencies sponsoring the program, but also those made by outside 
organizations responsible for providing certain components (such as occupational sicills training In 
the three sequential^rokered sites). Furthermore, in sites where goods and services that affected 
the nature of the program treatment were donated to the sponsoring agency, the analysis 
estimated the market value of those contributions and counted it, too, as a program cost.^ For 
these reasons, the costs presented here may differ from those reflected in a sponsoring agency's 
own fiscal records. 

This appendix does not present estimates of the cost of education and training services 
received by members of the control group. Thus, it provides no insights into the incremental 
investment that the JOBSTART sites made for the experimental group. An estimate of these 
incremental or "net" costs would be an important part of a full benefit-cost analysis, where the 
value of experimental-control differences in earnings and other outcomes (the •l)enefits") is 
compared with the experimental-control differences in the cost of services producing those 
benefits. As Chapter 4 showed, a substantial proportion of the control group did receive 



ipor example, at Chicago Commons, many of the basic supplies e^isemial to operatmg some of he 
training courses were donatal to the program. The estimated value of these supplies, as reported in the 
Ws ?nS audh report, was thuf counted as a program cost. As another illustration the hfe skills 
XSps rbotrchicago bommons and Connelley in Pittsburgh were conducted free of charge a the 
proS o^rganizations. These donated services were thus valued and include, m the ota 

Sst Their Jimated value was based on the number of sessions conducted and by a proxy value of the 
average cost per session to the agency providing the service. 
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education and training sendees during the research follow-up period, which means that the net 
cost of JOBSTART in some sites may be considerably smaller than the gross cost reported here. 
A. Data Sources and Accounting Periods 

Data for the cost analysis were gathered from a variety of sources. These include: 

• individual staff salary information; 

• site expenditure reports, which showed overall expenditures on salaries and 
fnnge benefits, rent, utilities, supplies, equipment, administration, and so on; 

• program enrollment and participation data covering JOBSTART and non- 
JOBSTART participants, both for the program as a whole and for individual 
components (such as education classes and training classes); 

• JTPA expenditure data in sites where JTPA f-mds were used to provide 
program services; 

• agency data on support service expenditures covering needs-based payments 
transportation, food, child care, and other participant payments; 

• interviews with program staff concerning the allocation of staff time across 
program components and between JOBSTART and non-JOBSTART 
functions, and other aspects of site operations that affected the use of 
resources; and 

' J^cJaJ^?^ °" experimental group's degree of participation in 
JUBJiTART activities. 

In most cases, data from these sources covered a one-year "steady-state" period sometime 
between 1985 and 1988 (depending on the site), the years during which JOBSTART was funded .2 
However, the actual calendar months of this accounting period varied according to each site's date 
of entry into the demonstration and the particular months covered by its annual fiscal reporting 
period.^ 

Ideally a steady-state period should reflect a time during which program operations are 
relatively stable. Although it was difficult to define such a period for JOBSTART because of 



JOBSTART program at Connelley in Pittsburgh and SER/Corpus Christi changed substantialiv 
from the first year of operations to the next. Consequently, cost and part7dpation dLffor'bmh y^l^s 2 

some sit^ participation data and expenditure reports did not cover exactly the same time oeriod 
so a number of additional adjustments had to be made in estimating average st Jd^staTex^endi^ur^' 
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the demonstration's relatively short duration, the period selected in most sites began at least 
several months after the initiation of the project (in order to avoid the start-up costs associated 
with beginning a new program), and ended at least several months prior to the termination of the 
demonstration (in order to exclude the phasedown period).'* To remove the influence of inflation 
resulting from the use of costs from different calendar periods in different sites, all estimates were 
inflated or deflated to 1986 dollars. 

B. Excluded Costs 

In estimating the average cost per JOBSTART experimental, adjustments were made to 
exclude two categories of expenditures embedded in the sites' fiscal data: (1) research-related 
costs, and (2) the costs of services or activities that were offered to or used by non-JOBSTART 
participants. A fraction of program expenditures during the steady-state period resulted 
exclusively from research requirements. These included the extra costs involved in recruiting and 
processing individuals who became part of the -ontrol group, as well as the costs of staff timr; 
spent on conducting ra.idom assignment, completing the research enrollment forms, and 
participating in research-related interviews with MDRC personnel. These activities are not part 
of the normal effort of operating a JOBSTART program. Thus, the resources spent on them are 
not counted in the average cost estimates reported here. 

Several of the sites also offered a number of services that were not a part of the 
JOBSTART program but were nonetheless captured in the agencies' aggregate expenditure 
reports. These, too, had to be excluded from the estimates of JOBSTART costs. This issue was 
most significant in the three Job Corps sites, where some Corpsmembers lived in dormitories at 
the centers, while others lived at home while attending Job Corps activities. All of the 
JOBSTART experimentals were nonresidents in these sites, and were thus unaffected by the 
services intended exclusively for the residents. These residential-only services included: dormitory 
provisions, most night-time and weekend recreational activities,^ and supervision by residential 
advisors and dormitory attendants. The share of total Job Corps costs associated with exclusively 



Because the core education and training services in most sitts were already in place prior to 
JOBSTART and continued after the demonstration, start-up and phasedown costs for the JOBSTART 
Demonstration were not an issue for these components. 

^Although nonresidents were invited to participate in all recreation activities, they did so much less 
frequently than residential Corpsmembers. 
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residential aspects of the program was thus estimated and eliminated from the resources counted 
in determining the cost of JOBSTART.^ 

C. Calculating *tig Average Cost per Experimental 

In each site, the total average cost of JOBSTART per experimental was determined by 
summing the average cost of several relatively distinct program components and services. 
Determining these component costs involved several steps. First, an average unit cost during 
the steady-state period - that is, the cost of serving one person in the component for a specified 
unit of time - was calculated. The unit of measure varied for some components, mostly 
depending on whether the activity operated on an open-entiy/open-exit or a fixed-cycle basis.'' 
Thus, for open-entiy components, the average cost of serving one person for one month in the 
activity was estimated; for fixed-cycle activities, the average cost per person who ever entered a 
given cycle of the activity was estimated. 

The numerator in these unit costs incorporated total expenditures for personnel and 
overhead functions, including expenses incurred for non-JOBSTART participants in sites where 
the experimentals were enrolled along with other persons in regular agency activities.^ The value 
of donated goods and services was also counted as a program expenditure. The denominator 
includes all participants (both JOBSTART and non-JOBSTART) in the component. Thus, for 
example, the unit cost of basic education at a site is the full cost of classes in which JOBSTART 
participants were enrolled, divided by the total number of students in the classes. 

The average unit cost was then multiplied by a conesponding participation measure.^ For 
open-entry components, the unit cost was multiplied by the average number of months in which 
experimentals spent any hours in the activity (including zero months for experimentals with no 



technical assistance project at BSA in New York City is another example of separate activities 
whose costs had to be excluded in estimating JOBSTART costs. 

^Vhen participants leave open-entry activities, they are typically replaced by other individuals. Thus, 
the average value of resources expended per person for these components varies with the length of time 
an average participant receives that service. However, if entry into a training class is based on a fixed cycle 
and a student who drops out is not replaced by another student, the costs for that student's "slot" are still 
incurred by the agency ca that student's behalf, regardless of his or her length of stay. 

Spor some activities, sites mainstreamed JOBSTART participants with non-JOBSTART partiapants. 
Other activities (for example, counseling, life skills instruction, basic education, or training services that 
were not normally provided to part of the agency's program) included JOBSTART participants only. See 
Auspos et al , 1989, for more details on the adaptations the sites made for JOBSTART. 

'In order to spread average unit costs among all experimentals and to cover the full penod of their 
involvement in the program, the participation measures captured participation that occurred at any time 
during the demonstration, not just within the steady-state period. 
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hours in the activity). ^ For components that operated on a fixed-cycle basis, the unit cost (the 
cost per person who entered the activity) was multiplied by the proportion of JOBSTART 
experimentals who ever entered the activity. The values for both types of participation 
measures were based on the experiences of the experimentals in the '^impact sample" (the sample 
of survey responders used in this report), not of all experimentals who were randomly assigned.^^ 

IL Ac counting for Site Variations in Average Costs 

The average total cosf of JOBSTARTs core components per experimental varied widely 
across the sites. (See Table E.l.) Although it fell within $4,500 to $6,500 in most sites, it ranged 
from less than $2,100 in CET/San Jose, EGOS in Denver, and SER/Corpus Christi to a high of 
about $7,500 in BSA in New York City. Several factors account for this diversity. Most notably, 
the sites differed in terms of both the amount of experimentals' participation in JOBSTART and 
the unit cost of that participation. This can be seen in Table E.1, which presents information for 
each site on the average number of months that the experimental group participated in program 
activities and the average monthly cost of participation.^^ Sites where the value of both of these 
variables was lower than in other sites were among the least expensive JOBSTART programs. 
For example, this combination of factors helps to explain why CET/San Jose (where experimentals 
participated for only 4.4 months and the average monthly cost was only $462) had the least 
expensive JOBSTART program. 

In several sites, higher unit costs were somewhat offset by shorter participation, yielding 
lower total average costs than were observed in some other sites. For example, the average 



^^Ls approach (that is, multiplying the average cost of serving one person for one month by the 
average number of months that JOBSTART experimentals spent in the component) allocates costs between 
JOBSTART and non-JOBSTART participants on the basis of their respective lengths of stay in the activity. 

^^This approach assumes that there wis no difference in the average cost of serving JOBSTART and 
non-JOBSTART participants who actually began the component. The costs were considered fixed, whether 
or not the students stayed until completion. No data were available for comparing the lengths of stay of 
JOBSTART and non-JOBSTART participants. 

^^ee Chapter 2 for a discussion of the sample used in this report. In general, the experimentals in 
this sample, on average, had slightly more months with any hours of participation than did all 
experimentals, and thus may have been slightly more expensive to serve than those not in this sample. 

^^e average monthly cost for each site - created to facilitate comparisons across sites - was 
calculated by dividing the average total cost (for experimentals) by the average number of months active 
in JOBSTART (for experimentals). Although this calculation assumes that the average total cost in all 
sites was variable, as noted earlier, the costs of fixed-<ycle activities were actually calculated on a fixed- 
cycle basis. 
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TABLE E.l 

AVERAGE MONTHLY AND TOTAL COSTS PER EXPERIMENTAL AND PER PARTICIPANT, BY SITE 



I 

ro 

to 
I 



Site 



Concurrent 

Atlanta Job Corps 
CET/San Jose 
Chicago Coninons 
Connelley (Pittsburgh) 
East LA Skills Center 
EGOS (Denver) 
Phoenix Job Corps 
SER/Corpus Chi-isti 

Sequential/ in-house 

El Centro (Dallas) 
LA Job Corps 

Sequen tial/b roke red 



Percent of Experimental s 

with Any Hours 

of Participation Experimentals Participants 



Average Number of Months 
Active in JOBSTART" 



84.80% 

64.00 

91.90 

98.90 

82.40 

93.55 

86.57 

98.30 



100.00 
79.10 



4.94 

4.40^ 

4.32 

^.16 

5.04 

6.86<^ 

6.25 

5.03 



5.25 
7.17 



5.82 

6.8e<= 

4.71 

9.26 

6.12 

7.33'' 

7.22 

5.12 



5.25 
9.05 



Average Cost per 
Month Active in 
JOBSTART 



$845 
462 
1,499 

566 
970 
303 
793 
417 



1,011 
774 



Average Total Cost 
for Core Components 
Experimentals Participants' 



$4,173 

2,034 
6,477 
5,185 
4,887 
2,076 
4,956 
2,098 



5,306 
5,550 



$4,921 
3,178 
7,048 
5,243 
5,931 
2,219 
5,725 
2,134 



5,306 
7,016 



Allentown (Buffalo) 

BSA (NYC) 

CREC (Hartford) 



100.00 
75.38 
88.89 



8.88 
4.77 
5.60 



8.88 
6.33 
6.30 



660 
1,569 
923 



SOURCE: 



MDRC calculations from site and MDRC participation, fiscal, and administrative data. 



5,862 
7,484 
5,166 



5,862 
9,928 
5,812 



NOTES: Estimates in this table used data for all experimentals for whom there were 24 months of follow-up survey data, including those who 
were assigned to JOBSTART but did not participate. "Participants" are the subset of experimentals who were active for at least one hour in any JOBSTART 
component within 24 months of random assignment. 

All costs are in 1986 dollars. 

"Unless otherwise stated, the number of months active in JOBSTART is defined as the number of months with hours in any JOBSTART 

component. 

These estimates were obtained by dividing the average total cost per experimental by the percentage of experimentals with any hours 

of participation. 

^For consistency with the definition of unit costs in this site, the number of months active in JOBSTART is measured from the month 
of random dssiQnmcnt to the Idst month with hours in Any component. 

^For consistency with the definition of unit costs in this site, the number of months active in JOBSTART is measured from the first 
month with hours in any JOBSTART component to the last month with hours in any component. 
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monthly cx)st of JOBSTART at the Atlanta Job Corps was higher than at Connelley in Pittsburgh 
($845 compared to $566), However, Atlanta's overall average cost was lower ($4,173 compared 
to $5,185) because its experimentals were active in the program for less time (4,94 months 
compared to 9,16 months). 

The wide variation in average monthly costs across the sites (ranging from $303 in ECrOS 
in Denver to $1,569 in BSA in New York City) has a number of sources. One is enrollment 
levels. For example, if the number of participants "on-board" a program in a typical month is 
high relative to the number of program instructors, the total monthly instructional costs (and the 
corresponding overhead expenditures) will be spread over many people, lowering the average unit 
cost per participant. This factor helps to account for the relatively low monthly cost of 
JOBSTART at EGOS in Denver, a large public vocational school with more than 15,000 students. 
In contrast, at BSA in New York City, high monthly costs were partly the result of its having 
enrolled only about half the number of students the school had the capacity to serve at any one 
time. Staffing decisions can also affect costs. For example, Chicago Commons assigned two 
instructors to all training classes, an unusual practice among the JOBSTART sites, and this raised 
its average monthly cost per participant. Differences in wage scales further explain some of the 
variation in monthly costs. As an illustration, the average hourly wage paid to instructors at 
SER/Corpus Christi was about half the hourly rate received by teachers at the East Los Angeles 
Skills Center. 

Differences in the scope of activities and services across the sites also account for 
differences in average monthly costs. For instance, as will be seen below, the three least 
expensive hitcs had no life skills or work-readiness instruction, and one of them (CET/San Jose) 
spent httle on support service payments and basic education as a separate activity. Differences 
in overhead costs, such as those for rent and administration, also varied across the JOBSTART 
sites. 

Table E.1 shows that in some sites a substantial proportion of experimentals left the 
program after random assignment and so never entered any program component. At the East 
Los Angeles Skills Center, for example, only 82 percent of the experimental group ever received 
JOBSTART services. One consequence of such attrition is that a site's average cost per person 
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actually served by the program is higher than its average cost per experimental}^ At the Los 
Angeles Job Corps, it was 26 percent higher ($7,016 compared to $5,550). Although the average 
cost per experimental would be the appropriate nmber to include in a benefit-cost analysis for 
the JOBSTART evaluation, the average cost per participant may be a better guide for 
administrators interested in the implications for an agency's budget of operating a JOBSTART 
program. 

in. Component Costs 

This section discusses how the costs of the individual JOBSTART components contributed 
to the total average cost at each oite and further illustrates the sources of variation in those total 
costs across the sites. 

A. Definitions of Components 

For purposes of the cost analysis, JOBSTART functions were divided into eight main 
components. The category of recruitment, intake, and orientation was defined to encompass sites' 
efforts to attract and enroll individuals into the JOBSTART program and to prepare them, 
through special presentations or workshops, for attending the agency's regular education and 
trainmg classes. This process involved screening applicants to determine whether they met all 
JOBSTART as well as JTPA or Job Corps eligibility criteria. Random assignment, special data 
collection, and the additional efforts devoted to recruiting and processing extra individuals to 
allow the creation of a control group also occurred during the recruitment and intake stages. 
previously mentioned, these latter activities were defined a* research-only costs, and hence they 
were not counted in the average cost of this component. 

Following orientation, experimentals in all sites were scheduled to attend basic education 
classes. Occupational <'ldlls training classes were offered concurrently or following the completion 



average cost per participant was calculated by dividing the average cost per experimental by the 
percentage of experimentals with any hours of participation. 

^^The costs included here for JTPA eligibility determination only cover a site's efforts to help 
applicants identify and collect the necessary documents and complete the required paperwork as pan of 
the application process. It generally does not include the time JTPA staff spent reviewing those documents 
and approving the applications. 
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of basic education. The costs of these two components were estimated separately, although in 
some sites the line between them was not sharp. 

Several sites also enriched their programs by offering work-readiness classes or life sJdlb 
workshops that covered topics such as work habits, health, and financial management. In addition, 
the Job Corps sites offered avocational classes in drivers education, sewing, and physical 
education. These were counted as part of the work-readiness/life skills component for the cost 
analysis. 

Job placement was defined to include instruction in job-seeking techniques as well as direct 
placement efforts. Coordination and counseling include staff efforts ti monitor participants' 
attendance and progress in JOBSTART activities and to counsel them on an as-needed basis. In 
a number of sites where JOBSTART was operated alongside other programs, a special counselor 
was designated to perform this function exclusively for the experimental group. 

Support services are defined as the special expenditures intended to help motivate 
participants to attend program activities regularly, or to help offset some of the potential barriers 
to attendance. The particular types of support services that were available varied across sites, but 
included payments for child care, transportation, needs-based payments, food, and attendance and 
achievement awards. 

Medical and dental semces were an additional component offered in the three Job Corps 
sites through an on-site clinic. To a much lesser extent, such services were also offered at EGOS 
in Denver through a formal agreement with an outside agency to which staff routinely referred 
participants.^^ These services are not considered to be part of the core JOBSTART model, 
however. Consequently, their costs have not been included in the total average costs reported 
above (although they have been estimated). 

It should be noted that the information used in allocating total site costs across components 
(such as the proportion of staff time spent on recruitment and intake versus counseling and 



i^Data limitations have precluded perfect consistency across all sites in the definition of each 
component. Especially problematic is the distinction between basic education and skills training at 
CEiySan Jose and Chicago Commons. To a large extent, basic education instruction in those two sites 
was integrated with occupational skills training. However, those sites also operated separate remedial 
education classes. The cost analysis counts only participation in those remediation classes as basic 
education. This definition of basic education is consistent with that used in the calculation of education 

hours, as reported in Chapter 3. . • 

I'^Some of the other JOBSTART sites also referred participants to such services at outside agencies, 
but on a much less formal basis. In these sites, the costs incurred by those agencies were not estimated. 
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coordination) was often imprecise. As a result, some component cost estimates are much less 
certain than others. However, this does not affect the estimate of any site's average tota'. cost. 
B- Vari ations in the Cost of TOBSTART romponents 

Table E.2 shows the estimated average cost per experimental of each JOBSTART 
component for each site. In addition, for the four sites where medical and dental services were 
provided, the table shows how the total average cost changes when expenditures for these services 
are counted. In the three Job Corps sites, these amounted to fairly sizable expenditures - $564 
per experimental at Los Angeles, $690 at Atlanta, and $357 at Phoenix. In addition, EGOS in 
Denver, which, through routine referrals, provided eyeglasses and dental examinations, spent $24 
per experimental on medical expenses. 

Recruitment, intake, and orientation activities accounted for between 9 and 13 percent of 
the total average cost of the core JOBSTART components in most of the sites, but reached as 
high as 19 percent at the East Lo5 Angeles Skills Center. In absolute value, BSA in New York 
City spent the most on these upfront efforts ($1,313), while SER/Corpus Christi spent the least 
($227 per experimental). 

Basic education, skills training, and coordination and counseling were usually among the 
most expensive components to operate across the sites. When taken together, they accounted 
for no less than 44 percent (the estimate for the Atlanta Job Corps) of the total average cost 
of the core JOBSTART components, and they reached as high as 86 percent (the estimate for 
SER/Corpus Christi). In most of the sites, this combination of functions accounted for at least 
two-thirds of the iotal average cost. 

Particularly notable are the high costs of basic education at the three sequential^rokered 
sites. At BSA in New York City and CREC in Hartford, basic education aJone accounied for 
half the total cost ($3,836 and $2,634 per experimental, respectively). Tb- cost of basic education 
was also high at Chicago Commons ($1,400), which added a separate education class specifically 
for JOBSTART participants, and at El Centro in Dallas ($1,301) and the East Los Angeles Skills 
Center ($1,114). OveraU, the percentage of total average costs devoted strictly to basic education 
ranged from 4 percent at CET/San Jose to 51 percent at BSA and CREC. Education costs were 
especially low at CET/San Jose ($88 per experimental) in part because most of the hours that 
experimentals spent in that site were spent in training classes, which, it should be recalled, also 
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TABLE E.2 

AVERAGE JOBSTART OPERATING COSTS PER EXPERIMENTAL, BY SIT^ 



I 











Concurrent 










Sequential/ 
In-House 


Sequential/ 
Broke d 


Component 


Atlanta 

Job Corps 


CET/ 

San 

Jose 


East LA 

Chicago Connelley Skills EGOS 
Comnons (Pittsburgh) Center (Denver 


Phoenix 
) Job Corps 


SER/ 

Corpus 

Christi 


El 

Centro 
(Dallas 


LA 

) Job Corps 


Al lentown 
(Buffalo) 


BSA 
(NYC) 


CREC 

(Hartford) 


Recruitment, intake, 
and orientation 


$602 


$245 


$327 


$445 


$923 


$245 


$645 


$227 


$568 


$586 


$328 


$1,313 


$689 


Basic education 


529 


88 


1 A nn 
1 ,400 


544 


1 114 

1,114 


384 


* 939 


632 


1,301 


648 


1,147 


3,836 


2,634 


Occupational 'Vills 
training 


283 


1,031 






1 mi 


907 


1,446 


533 


1,175 


1,478 


529 


453 


332 


Work-readiness or 
life skills 


839 


n/a 


35 


283 


n/a 


n/a 


343 


n/a 


392 


442 


1,438 


920 


n/a 


Job development and 
placement assistance 


92 


308 


262 


334 


36 


19 


188 


73 


639 


302 


628 


n/a 


n/a 


Counseling and program 
coordination 


1,031 


301 


1,159 


2,239 


1,196 


664 


498 


633 


719 


705 


1,058 


757 


1,279 


Support services* 


797 


61 


363 


447 


87 


467 


897 


593*^ 


512 


1,389 


731 


205 


232 


Subtotal for core 
JOBSTART components 


4,173 


2,034 


6,477 


5,185 


4,887 


2,076 


4,956 


2,098 


5,306 


5.550 


5.862 


7,484 


5.166 


Medical/dental^ 


690 


n/a 


n/a 


n/a 


n/a 


24 


357 


n/a 


n/a 


564 


n/a 


n/a 


n/a 


Total 


4,863 


2,034 


6,477 


5,185 


•4,887 


2,100 


5,313 


2,098 


5,306 


6,114 


5,862 


7,484 


5,166 



SOURCE: MORC calculations from site and MORC participation, fiscal, and administrative data. 



NOTES: Estimates in this table used data for all experimentals for whom there were 24 months of follow-up survey data, including those who were 
assigned to JOBSTART but did not participate. 

All costs are in 1986 dollars. 

^Includes such costs as needs-based and incentive payments; transportation, child c;^re, and clothing allowances; and food. 

'^Because of data limitations, the support services cost for JOBSTART and non-JOBSTART participants in this site could not be separated from 
other expenditures in the general overhead rate used in estimating the cost of the other program components. Thus, the per-experimental cost of each component 
inch'des the cost for support services. To avoid double-counting this expenditure in the average total cost per experimental, the $593 estimated value of 
support services, which was calculated from indi vi dual -level data available only for JOBSIARI youths, is not included in the sum of component costs. 
Q ^These services were routinely available in and measured for the Job Corps sites and EGOS only. 
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included some work on basic education skills. (Basic education that occurred in the context of 
occupational skills training was counted as skills training.) 

The resources spent on skills training also varied widely across the sites. Not surprisingly 
as a proportion of total average costs, expenditures for this component were lowest at the 
sequentiaVbrokered sites (accounting for less than 10 percent of those costs), where only about 
one-quarter of the experimentals made the transition to a training activity. In the other sites, this 
component accounted for between 7 percent and 51 percent of total costs. In absolute value, 
it was most expensive at Chicago Commons ($2,931), where the training classes were small and 
operated on a fixed-cycle basis (dropouts within a cycle were usually not replaced with other 
students). The classes also involved fairly technical instruction and used significant amounts of 
purchased and donated supplies. 

Across all sites, the per-experimental cost of coordination and counseling ranged from $301 
at CET/San Jose to $2,239 at Connelley in Pittsburgh, where it accounted for an unusually high 
43 percent of the total average cost per experimental. In contrast, job search assistance and 
placement in roost sites accounted for no more than 6 percent of total average costs, although 
it ranged from $19 per experimental at EGOS in Denver to $639 at El Centro in Dallas. In 
most sites, this tended to be a relatively inexpensive component because relatively few staff 
members were usually assigned to instruct or assist many different participants. (In two of the 
three sequentiaVbrokered sites, job search assistance was not offered by the sponsoring agencies 
but, instead, was a function expected to be perf'^nned by the outside training vendors.) 

The sites also varied widely in their expenditures on work-readiness and life skills instruction. 
Indeed, some sites did not offer these activities at all, while others gave them considerable 
emphasis. Allentown in Buffalo, for example, spent more per experimental on life skills than it 
did on education ($*,438, or 25 percent of total average costs). 

Expenditures on support service costs ranged from $61 per experimental at CET/San Jose 
to $1,389 (or 25 percent of the total average cost) at the Los Angeles Job Corps. The other Job 
Corps sites, as well as SER/Corpus Christi, also devoted a relatively high proportion of resources 
to these services - about one-fifth to more than one-fourth of total average costs. 

Table E.3 shows the breakdown of support services costs for child care, transportation, 
food, needs-based payments, and other purposes (which included incentives for attendance and 
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TABLE E.3 



AVERAGE JOBSTART SUPPORT SERVICES AND PARTICIPANT PAYMENT COSTS PER EXPERIMENTAL, BY SITE 











Concurrent 










Sequential/ 
In-House 


Sequentia 
Brokered 


1/ 


Support Service 


Atlanta 
Job Corps 


CET/ 

San 

Jose 


Chicago 
Coffwnons 


East LA SER/ 
Connelley Skills EGOS Phoenix Corpus 
(Pittsburgh) Center (Denver) Job Corps Christi 


El 

Centro 
(Dallas 


LA 

) Job Corps 


A1 1 en town 
(Buffalo) 


BSA CREC 
(NYC) (Hartford) 


Child care 


n/a 


n/a 


n/a 


$107 


$21 


$339 


n/a 


$86 


n/a 


n/a 


$276 


$45 


$27 


Transportation 


n/a 


n/a 


n/a 


n/a 


12 


121 


n/a 


n/a 


$161 


$206 


86 


107 


168 


Food 


$153 


n/a 


n/a 


n/a 


17 


7 


$221 


n/a 


n/a 


259 


n/a 


n/a 


n/a 


Needs-based and incentive 
payments, clothing allow- 
ances, and miscellaneous 


644 


$61 


$363 


340 


37 


0 


676 


507 


3'il 


924 


372 


53 


37 


Total 


797 


61 


363 


447 


87 


467 


897 


593 


512 


1,389 


734 


205 


232 



SOURCE: MDRC calculations from site and MDRC participation, fiscal, and administrative data. 



NOTES: Estimates in this table used data for all experimentals for whom there were 24 months of follow-up survey data, including those who 
assigned to JOBSTART but did not participate. 

All costs are in 1986 dollars. 
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achievement). Almost all the sites offered needs-based payments, while about half of them paid 
for child care and transportation. Food costs were substantial in the three Job Corps sites 
(ranging from $153 to $259 per experimental), where the on-site cafeterias offered regular meals 
to both residential and nonresidential participants. 



^^All Job Corps participants (including those in JOBSTART) received a $40 monthly living allowance, 
which increased to $60 after they were active for 61 ("good") days, and increased again to $80 after they 
were active for 181 days. After that, participants were eligible to receive a merit pay level of $90 to $100 
per month. In addition, upon termination ft^om the Job Corps, those who remained in the program for 
a specified length of time received a separate "readjustment allowance" for each month of participation. 
This allowance ranged from $75 to $100 per month depending on the number of days they were active. 
Participants could have a portion of this allowance sent to a dependent family member. If they made that 
choice, the Job Corps contributed an equal amount to the family member 
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APPENDIX F 
SUPPLEMENTAL TABLES TO CHAPTERS 4 AND 5 



TABLE r.l 

IMPACTS ON MONTHLY RECEIPT OF EDUCATION OR TRAINING 



Outcome and Fol low-Up Period 


Experimental 


Controls 


Difference 




Received education or training 










Month 1 


68.6V 


5.4% 


63.1*** 


0.000 


Month 2 


82.6 


8.0 


74,6*** 


0.000 


Month 3 


75.4 


10.0 


65.4*** 


0.000 


Month 4 


67.5 


11.1 


56.3*** 


0.000 


Month 5 


60.7 


11.7 


49.0*** 


0.000 


Month 6 


52.4 


12.1 


40.4*** 


0.000 


Month 7 


43.5 


13.5 


30.0*** 


0.000 


Month 8 


35.3 


12.7 


22.6*** 


0.000 


Month 9 


32.0 


13.8 


18.2*** 


0.000 


Month 10 


28.6 


14.6 


14.0*** 


0.000 


Month 11 


24.5 


14.6 


9.9*** 


0.000 


Month 12 


19.3 


12.5 


6,8*** 


0.000 


Month 13 


21.3 


15.6 


5.7*** 


0.001 


Month 14 


17.2 


12.0 


5.2*** 


0.001 


Month 15 


16.6 


11.1 


5.6*** 


0.000 


Month 16 


15.5 


10.9 


4 , 5*** 


0.003 


Month 17 


14.8 


11.3 


3.6** 


0.022 


Month 18 


13.8 


11.1 


2.7* 


0.075 


Month 19 


13.2 


11.1 


2.0 


0.183 


kl^^ Aim. OA 

Month cO 


12 . 1 




-0, 1 


0. 968 


Month 21 


12.3 


12.6 


-0.2 


0.881 


Month 22 


11.7 


12.8 


-1.1 


0.473 


Month 23 


11.3 


11.3 


0.0 


0.982 


Month 24 


11.2 


10.3 


0.8 


0.559 


Sample size 


949 


890 







SOURCE: MDRC calculations from JOBSTART enrollment form, MIS, and survey data. 



NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

"Education or training" includes JOBSTART and non-JOBSTART education, 
occupational skills training, and related activities. 

Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted 
means from linear analysis of covariance procedures controlling for up to 31 kinds of 
difference in characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987; 
and Appendix Table B*4). There may be slight discrepancies in reported sums and differences 
of these adjusted means because of rounding. 

*The column labeled "p" is the statistical significance level of the 
difference between experimental and control group outcomes: that is, p is the probability 
that average outcomes are different only because of random error. A two-tailed t-test was 
applied to each difference between average experimental and control group outcomes. 
Statistical significance levels are indicated as *** « 1 percent; ** = 5 percent; * = 10 
percent. 
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TABLE Fe2 

IMPACTS ON MONTHLY AVERAGE HOURS OF EDUCATION OR TRAINING RECEIVED 



Outcome and Fo11ow-Up Period 



Experi mentals Controls Difference 



Hours of education or 
training received 



Month 1 




9 CI 


Z6.93*** 


0.000 


Month 2 


fl<i fld 
w 7 • OH 


V •Hi 


63.43*** 


0.000 


Month 3 


0 J •10 


O. 3/ 


54. 59*** 


0.000 


Month 4 


52.57 


in di 




0,000 


Month 5 


HO • do 


1 1 r\c 
1 1 .Ud 


35.20*** 


0.000 


Month 6 


J 7 . H J 


1 1 00 


27.44*** 


0.000 


Month 7 


11 

J 1 • 911 


19 01 


18.68*** 


0.000 


Month 8 


c/ . yjc 


1 o no 


14, 10*** 


0.000 


Month 9 




1 A A^ 
14.41 


10.38*** 


0.000 


Month 10 




1 A 1 C 
14.13 


7 .82*** 


0.000 


Month 11 




19 01 
1^ • ^1 


A Ji n ** 

4.43** 


0.012 


Month 12 




11.79 


3.81** 


0.031 


Month 13 


16.60 


10.78 


5.82*** 


n nni 

u . UU 1 


Month 14 


18.66 


10.50 


8.16*** 


0.000 


Month 15 


18.42 


10.00 


8.41*** 


0.000 


Month 16 


17.86 


9.71 


8.1b*** 


0.000 


Month 17 


15.72 


9.77 


5.95*** 


0.001 


Month 18 


15,40 


9.38 


6.01*** 


0.001 


Month 19 


14.47 


9.63 


4.84*** 


0.007 


Month 20 


13.56 


9.83 


3.73** 


0.036 


Month 21 


13.36 


10.47 


2.89 


0.111 


Month 22 


12.49 


10.54 


1.95 


0.281 


Month 23 


12.39 


10.02 


2.37 


0.187 


Month 24 


11.21 


9.45 


1.76 


0.290 


Sample size 


949 


890 






SOURCE: 


MDRC calculations from JOBSTART 


enrol Iment form, 


MIS, and survey 


data. 



NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

For experimental s, "hours of education or training" include JOBSTART hours 
from MIS data and non-JOBSTART hours from survey data. 

Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted 
means from linear analysis of covariance procedures controlling for up to 31 kinds of 
difference in characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987: 
and Appendix Table B.4). There may be slight discrepancies in reported sums and dif/erences 
of these adjusted means because of rounding. 

*The column labeled "p" is the statistical significance level of the 
difference between experimental and control group outcomes: that is, p is the probability 
that average outcomes are different only because of random error. A two-tailed t-test was 
applied to each difference between average experimental and control group outcomes. 
Statistical significance levels are indicated as *** » 1 percent; ** « 5 percent; * « 10 
percent. * 
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TABLE F.3 



IMPACTS ON MONTHLY RECEIPT OF BASIC EDUCATION 



Outcome and Fol low-Up Period 


Experimental s 


Control s 


u 1 T T erence 


na 
P 


Received basic education 










Month 1 


47.8% 


3.1% 


£\ £\ f n n n 


ft Aftft 

u.uuu 


Month 2 


78.9 


3.7 


7b . d^^^ 


u . uuu 


Month 3 


70.3 


4.9 


65. 5*** 


ft ft ftft 

0.000 


Month 4 


59.1 


5.5 


53 . 0 


0.000 


Month 5 


50.3 


6.5 


43 . o 


r\ f\r\r\ 

0.000 


Month 6 


36.1 


7.3 


28.8*** 


0.000 


Month 7 


29.0 


7.9 


21.1*** 


0.000 


Month 8 


23.0 


8.0 


15 . 0 


0,000 


Month 9 


.18.7 


8.6 


10 . 1*** 


0.000 


Month 10 


15-5 


8.6 


K y w WW 


0.000 


Month 11 


14.4 


8.3 


6 , 1*** 


0.000 


Month 12 


11.3 


7.2 


^ ^ A A # 


0.002 


Month 13 


13.9 


10.3 


3.6** 


f\ AIT 

0.01/ 


Month 14 


12.3 


8.4 


3 . g***^ 


0 .006 


Month 15 


11.5 


7.1 




0.001 


Month 16 


10.8 


6.8 


A U 


0 .003 


Month if 


If) 7 


7 . 1 


3.1** 


0.017 


Month 18 


9.0 


7.2 


1.8 


0.155 


Month 19 


8.8 


7.1 


1.6 


0.193 


Month 20 


7.8 


7.1 


0.7 


0.572 


Month 21 


7.7 


7.8 


-0.1 


0.952 


Month 22 


7.4 


8.1 


-0.7 


0.570 


Month 23 


7.6 


7.0 


0.6 


0.652 


Month 24 


7.4 


6.3 


1.1 


0.352 


Sample size 


949 


890 







SOURCE: MDRC calculations from JOBSTART enrollment form, MIS, and survey data. 



NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

"Basic education" includes JOBSTART and non-JOBSTART education activities. 

Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted 
means from linear analysis of covariance procedures controlling for up to 31 kinds of 
difference in characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987; 
and Appendix Table B.4). There may be slight discrepancies in reported sums and differences 
of these adjusted means because of rounding. 

*The column labeled "p" is the statistical significance level of the 
difference between experimental and control group outcomes: that is, p is the probability 
that average outcomes are different only because of random error. A two-tailed t-test was 
applied to each difference between average experimental and control group outcomes. 
Statistical significance levels are indicated as *** « 1 percent; ** = 5 percent; * = 10 
percent. 
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TABLE F.4 

IMPACTS ON MONTHLY RECEIPT OF TRAINING 



Outcome and Fo11ow-Up Period 



Experimental s Controls Difference 



Received 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 
Month 



training 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 



Sample size 



30.7V 


3 8V 




0.000 


47.4 


6 n 

u • u 


d1 Aifkir 

HI , IJ^ww 


0.000 


47.8 


7 6 


An Oiririr 


0.000 


45.7 


fi 8 
o • o 


17 n-kitir 

, u 


0.000 


44.2 




JO ♦ 1 


0.000 


41.7 






0.000 


34.6 






0.000 


28.8 


9 8 




0.000 


26.4 


10 7 




0.000 


24.1 




1 1 niririr 


0.000 


18.3 


11 5 


0 • o 


0.000 


15.5 


10 0 




0.000 


16.3 


12.2 


4.6*** 


0 005 


12.6 


8.1 


4.5*** 


0.002 


12.1 


7.8 




0.002 


11.1 


8.0 


3.1** 


0.020 


10.7 


8.3 


2.4* 


0.076 


10.6 


8.3 


2.3* 


0.089 


9.8 


8.2 


1.6 


0.219 


9.0 


9.1 


0.0 


0.973 


9.1 


9.3 


-0.1 


0.928 


8.6 


8.9 


-0.3 


0.824 


8.5 


8.0 


0.4 


0.734 


8.2 


7.5 


0.7 


0.592 


949 


890 







SOURCE: MDRC calculations from JOBSTART enrollment fonti. MIS, and survey data. 

NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

"Training" includes JOBSTART and non-JOBSTART occupational skills training 
activities. ' 

Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted 
means from linear analysis of covariance procedures controlling for up to 31 kinds of 
difference in characteristics before random assignment (see Ostle. 1975. p. 461; Cave. 1987- 
and Appendix Tab e B.4). There may be slight discrepancies in reported sums and diff;rence; 
of these adjusted means because of rounding. 

*The column labeled "p" is the statistical significance level of the 
difference between experimental and control group outcomes: that is. p is the probability 
that average outcomes are different only because of random error. A two-tailed t-test was 
applied to each difference between average experimental and control group outcomes. 
Statistical significance levels are indicated as *** « 1 percent: ** « 5 oercent- * « 10 
percent. * 
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TABLE F.5 



IMPACTS ON MONTHLY EMPLOYMENT RATES, 
BY GENDER AND PARENTAl STATUS 



Hen 



roi 1 ow*^up rcriou 


Pv npt^i mont a 1 c 
bApci iiiicriioi9 


Pont »»o 1 c 


u 1 I I c r Ciiwc 


P 


uVci eiiip 1 ujrcu 










Mnnth 1 


Ifi R 


27 7 


-11 2*** 


u . uuu 


Month 7 


21 1 


12 7 


-11 4*** 


0 Qno 

U . UUw 


Month 3 


24 0 


36 9 


-12 9*** 


0 nnn 


Month 4 


2fl 3 


41 6 


-13 


n nnn 

u • uuu 


Month 5 


29.6 


44 2 




n onn 


Month 6 


33 0 


44.0 


-11 O'*** 

1 1 « u 


n nni 


Month 7 

llwllWII * 


37.6 


44 8 


-7 2*** 




Month 8 


38 8 

<J u • u 


45 7 


-7 0** 


n ni4 


Month 9 


41 Q 


4Q 1 


-7 


n n27 


Month 10 


47 0 


R2 n 


-4 Q 


n 1 ifi 


Month 11 


4fi 0 
t u • u 


e;a ft 


-ft ft**'* 


n nnft 


Month 12 




OH . 1 


-1 2 


n m 


Month 11 
nun L 1 1 1 J 


DO • J 


DO . 0 


"U . D 


U . 0/ 0 


Month Id 
nun b 1 1 i*t 


2 


Aft 7 
HO . / 


-n fi 

"U . u 


n ftAft 


Month m 
nun 111 13 


^7 4 


DU • D 


7 n 


n c AO 

U. DOU 


Month 16 


52.6 


51.5 


1.1 


n.742 


Month 17 


51.9 


54.0 


-2.2 


0.513 


Month 18 


54.2 


55.8 


-1.7 


0.608 


Month 19 


54.1 


57.1 


-3.0 


0.368 


Month 20 


56.7 


57.0 


-0.3 


0.932 


Month 21 


57.1 


57.5 


-0.4 


0.913 


Month 22 


59.8 


60.5 


-0.8 


0.819 


Month 23 


58.0 


60.7 


-2.7 


0 42'? 

U . *V b «J 


Month 24 


59.7 


62.9 


-3.2 


0 339 

U . >^ J 7 


SamDl e si 7P 


438 


433 






ipn livino with nwn rhildfrpn) 










Pvpr 9mnlQ\/pd 










Month 1 


6.1 


8 4 


-2.3 


0 340 

U . J *f u 


Month 2 


5 9 


1 1 7 
11./ 


_5 8** 


0 026 

U . Ub u 


Month 3 

1 1 VI 1 k 1 1 «^ 


7 9 


13 4 


-5 5* 


n n^4 


Month 4 


9.6 


15 4 


-5.8* 


0 058 

W • u ^ u 


Month 5 

nui 1 k 1 1 »j 


12.2 


16 0 


-3.8 


0 212 


Month 6 

IIVIIvll V 


15.6 


17.0 


-1.4 


0.678 


Month 7 


18.2 


19.0 


-0.8 


0.827 


Month 8 


22.2 


19.9 


2.3 


0 511 


Month 9 


24.6 


22 4 


2.1 


0 574 


Month 10 


26.1 


23.0 


3.2 


0.423 


Month 1) 


24.9 


22.9 


2.0 


0.609 


Month 12 


25.6 


23.5 


2.0 


0.611 


Month 13 


30.8 


23.5 


7.4* 


0.073 


Month 14 


24.8 


18.4 


6.3* 


0.096 


Month 15 


24.7 


22.3 


2.4 


0. 526 


Month 16 


25.9 


21.1 


4.8 


0.210 


Month 17 


26.1 


22.9 


3.2 


0.414 


Month 18 


26.2 


23.3 


2.9 


0.460 


Month 19 


26.9 


24.3 


2.6 


0.525 


Month 20 


25.6 


24.8 


0.8 


0.846 


Month 21 


23.5 


25.7 


-2.2 


0.583 


Month 22 


23.1 


25.8 


-2.7 


0.498 


Month 23 


23.3 


25.1 


-1.8 


0.657 


Month 24 


23.0 


24.1 


-1.1 


0.783 


Sample size 


250 


234 
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TABLE F,5 (continued) 



Subgroup, Outcome, 

and FoUow-Up Period Experimental s Controls Difference 

Uomen not living with own child(ren), 
including those who did not have any 



Ever employed (%) 



Month 


1 


12 a 




.7 


A nil 


Month 


2 


17.0 






U«Odo 


Month 


3 


17.5 


?4 4 


■u • 7 


U« Ubo 


Month 


4 


17,6 


27 0 


r Q d** 


A ni 1 
U«Uli 


Month 


5 


22,3 


1? 1 


.Q fl^^ 
■7 • o 


n ni /I 


Month 


6 


24,3 


34 4 


-in 1** 


n m >! 
U #1114 


Month 


7 


28,8 




-7 «l* 


U •OoO 


Month 


8 


30,4 


38. Q 


"O « 3 


A AC 1 


Month 


9 


33.7 


34 fi 




A 0*3 A 


Month 


10 


35.0 


34 4 


U « D 


U.883 


Month 


11 


37 1 


JU « o 


n 7 

U • J 


0*940 


Month 


12 


3fi 5 


30 7 


-1 • 1 


0.794 


Month 


13 


45.2 


43.5 


1.8 


0.693 


Month 


14 


37.4 


32.1 


5.3 


0.211 


Month 


15 


38.1 




£ . *r 


n coo 


Month 


16 


37.7 


38.4 


-0.7 


0,878 


Month 


17 


36.9 


39.8 


-2.9 


0.506 


Month 


18 


37.4 


40.1 


-2.7 


0.543 


Month 


19 


37.0 


40.1 


-3.1 


0.498 


Month 


20 


36.4 


37.6 


-1.2 


0.794 


Month 


21 


44.7 


36.9 


7.8* 


0.086 


Month 


22 


45.3 


33.6 


il.7*** 


0.010 


Month 


23 


45.6 


39.0 


6.5 


0.151 


Month 


24 


44.1 


40.3 


3.8 


0.406 


Sample size 


261 


223 







SOURCE: MDRC calculations from JOBSTART enrollment form and survey data, 

NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on tr.e last day of the month. 

Average experimental and control group outcomes reported here are adjusted 
means from split-file linear analysis of covariance procedures controlling for up to 29 
kinds of difference in characteristics before random assignment (see Ostle, 1975, p, 461; 
Cave, 1987; and Appendix Table 8,4), There may be slight discrepancies in reported sums and 
differences of these adjusted means because of rounding. 

^The column labeled "p" is the statistical significance level of the 
difference between experimental and control group outcomes: that is, p is the probability 
that average outcomes are different only because of random error, A two-tailed t-test was 
applied to each difference between average experir. ntal and control group outcomes- 
Statistical significance levels are indicated as = 1 percent; s 5 perrent; * « 10 
percent. 
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TABLE F.6 



IMPACTS ON MONTHLY RATES OF POSITIVE ACTIVITY, 
BY GENOEK AND PARENTAL STATUS 



Subgroup I Outcome, 
and Fol low-Up Period 



Exoeri mentals Controls 



Difference 



Men 



Employed or 
or training 



in education 



Month 1 


75.7 


29.4 


46.3*** 


0.000 


Month 2 


89.2 


^5. 5 


53.7*** 


0.000 


Month 3 


85.5 


40.0 


45.5*** 


0.000 


Month 4 


80.2 


45.0 


35.2*** 


0.000 


Month 5 


76.5 


49.4 


27.2*** 


0.000 


Month 6 


71.9 


50. 1 


21.7*** 


0 000 


Month 7 


67.6 


51.2 


16 .4*** 


0 000 


Month 8 


61.9 


52.0 


0 0*** 

7.7 


0 00'\ 


Month 9 


64.2 


55.4 


8.7*** 




Month 10 


65.4 


58.6 


6.8** 


0 fT^Q 

U • U J 7 


Month 11 


60.7 


61.7 


-1.0 


0.757 


Month 12 


60.3 


59.1 


1.3 


0.702 


Month 13 


68.6 


66.0 


2.6 


0.408 


Month 14 


59.4 


54.9 


4.5 


0.180 


Month 15 


61.8 


55.3 


6.5** 


0.048 


Month 16 


62.6 


55.6 


7.1** 


0.029 


Month 17 


62.3 


58.5 


3.8 


0.236 


Month 18 


63.1 


59.5 


3.5 


0.279 


Month 19 


63.4 


61.7 


1.7 


0.592 


Month 20 


65.0 


61.5 


3.5 


0.279 


Month 21 


65.7 


61.7 


4.0 


0.210 


Month 22 


67.1 


66.3 


0.8 


0.812 


Month 23 


64.6 


65.8 


-1.2 


0.703 


Month 24 


64.6 


67.3 


-2.7 


0.396 


Sample size 


438 


433 







Uomen living with own child (ren) 

Employed or in education 
or training (%) 



Month 1 


71.6 


12.8 


58.8*** 


0.000 


Month 2 


84.4 


16.2 


68.2*** 


0.000 


Month 3 


75.0 


21.2 


53.8*** 


0.000 


Month 4 


71.1 


24.4 


46.7*** 


0.000 


Month 5 


63.7 


25.6 


38.1*** 


0.000 


Month 6 


59.7 


26.8 


32.8*** 


0.000 


Month 7 


53.9 


30.0 


24.0*** 


0.000 


Month 8 


53.4 


.'U.O 


22.5*** 


0.000 


Month 9 


51.5 


■^7.3 


14 .2*** 


0.002 


Month 10 


50.3 


36.8 


13.5*** 


0.003 


Month 11 


48.6 


36.9 


11.7*** 


0.010 


Month 12 


44.7 


35.2 


9.5** 


0.036 


Month 13 


48.6 


36.1 


12.5*** 


0.007 


Month 14 


39.0 


30.6 


8.4* 


0.060 


Month 15 


39.0 


32.2 


6.8 


0.130 


Month 16 


40.0 


30.8 


9.2** 


0.040 


Month 17 


39.2 


32.0 


7.2 


0.113 


Month 18 


40.0 


30.8 


9.2** 


0.040 


Month 19 


39.2 


31.7 


7.5* 


0.094 


Month 20 


35.6 


35.0 


0.6 


0.889 


Month 21 


33.8 


36.5 


-2.7 


0.545 


Month 22 


33.0 


38.2 


-5.2 


0.236 


Month 23 


33.3 


36.6 


-3.3 


0.459 


Month 24 


34.0 


35.0 


-1.0 


0.813 


Sample size 


250 


234 
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TABLE F.6 (continued) 



Subgroup, Outcome, 
and Fol low-Up Period 



Experimental s Controls 



.'ifference 



h'ointn not living with own child(ren), 
including those who did not have any 

Employed or in education 
or training (%) 
Month 1 



2 
3 
4 

5 
6 
7 



Month 
Month 
Month 
Month 
Month 
Month 
Month 8 
Month 9 
Month 10 
Month 11 
Month 12 
Month 13 
Month 14 
Month 15 
Month 16 
Month 17 
Month 18 
Month 19 
Month 20 
Month 21 
Month 22 
Month 23 
Month 24 

Saipple size 



70.5 


28.7 


n T uwww 


U .UUU 


84.8 


36.2 


48.6*** 




81.2 


38.6 


42.7*** 


^ AAA 

0 .000 


76.1 


42.3 


33.8*** 


0.000 


75.5 


45.3 


30.2*** 


0.000 


68.8 


47.7 


21.1*** 


0.000 


64.1 


52.4 


11.7*** 


0.009 


60.2 


53.3 


7.0 


0.128 


61.0 


50. 1 


11.0** 


0.017 


57.8 


49.4 


8.4* 


0.070 


57.7 


51.7 


6.0 


0.191 


55.4 


53.6 


1.8 


0.689 


64.2 


56.7 


7.6* 


0.093 


53.7 


42.1 


11.7*** 


0.010 


52.4 


45.4 


6.9 


0.134 


50.2 


48.0 


2.2 


0.632 


50.2 


50.2 


0.0 


0.996 


49.9 


51.1 


-1.2 


0,798 


48.9 


53.1 


-4.2 


0.376 


45.8 


50.9 


-5.1 


0.280 


54.2 


48.7 


5.5 


0.239 


53.7 


44.8 


9.0* 


O.Oi" 


53.1 


47.7 


5.4 


0.252 


51.0 


50.1 


0.9 


0.846 


261 


223 







SOURCE: MDRC calculations from JOBSTART enrollment form, MIS, and survey data. 

NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

"Positive activity" includes JOBSTART and non-JOBSTART education, 
occupational skills training, and related activities, as well as employment. 

Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted 
means from linear analysis of covariance procedures controlling for up to 31 kinds of 
difference in characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987, 
and Appendix Table B.4). There may be slight discrepancies in reported sums and differences 
of these adjusted means because of rounding. , * 

"The column labeled "p" is the statistical significance level of the 
difference between experimental and control group outcomes: that is, p is the probability 
that average outcomes are different only because of random error. A two-tailed t-test was 
applied to each difference between average experimental and control group outcomes. 
Statistical significance levels are indicated as *** » 1 percent; ** = 5 percent; * = 10 
percent. 
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APPENDIX G 
SUPPLEMENTAL TABLES TO CHAPTER 6 



TABLE G.l 

EXPERIMENTAL-CONTROL DIFFERENCE IN SERVICE RECEIPT. BY SITE 



Ever Received ^ny Education 
Sample or Training, Months 1-24 
Size Experlmentals Controls Difference 



Concurrent 



Atl anta Job Corps 


61 


90.4% 


47.6% 


42.8*** 


0.000 


CET/San Jose 


152 


71.7 


24.0 


47.7*** 


0.000 


tnicago tonrnons 


74 


96.3 


60.7 


35.6*** 


0.000 


Lonnei ley vPittsburgh; 


184 


99.4 


45.9 


53.5*** 


0.000 


East LA Skills Center 


100 


93.0 


57.8 


35.1*** 


0.000 


EGOS (Denver) 


183 


95.4 


50.0 


45.4*** 


0.000 


Phoenix Job Corps 


130 


90.3 


38.6 


51.7*** 


0.000 


SER/Corpus Christl 


236 


101.2 


34.8 


66.4*** 


0.000 


Sequential /in-house 












El Centro (Dallas) 


155 


98.8 


26.9 


71.9*** 


0.000 


LA Job Corps 


218 


85.3 


43.4 


41.9*** 


0.000 


Sequential /brokered 












Allentown (Buffalo) 


140 


100.5 


70.2 


30.4*** 


0.000 


BSA (NYC) 


119 


85.4 


52.1 


33.3*** 


0.000 


CREC (Hartford) 


87 


93.2 


48.3 


44.8*** 


0.000 


All sites 


1,839 


92.7 


44.2 


48.4*** 


0.000 



SOURCE: MDRC calculations from JOBSTART enrollment fomij MIS. and survey data. 



NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

"Any education or training" includes JOBSTART and non-JOBSTART education, 
occupational skills training, and related activities. 

Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group o'ltcomes reported here are adjusted 
means from two-way analysis of covariance procedures controlling for 19 kinds of difference 
in characteristics, other than site, before random assignment. The two categories used as 
factors were research assignment and site (see Ostle, 1975, p. 454). "All sites" outcomes 
are from a linear analysis of covariance procedure for the full sample controlling for up to 
31 kinds of difference in characteristics before random assignment (see Ostle, 1975, p. 461; 
Cave, 1987; and Appendix Table B.4). There may be slight discrepancies in reported sums and 
differences of these adjusted means because of rounding. 

^A two-tailed t-test was applied to each within-site impact. An F-test was 
applied to the interaction between site and experimental or control status. The p-value of 
the F-statistic is the probability that site impacts are different only because of random 
error. Statistical significance levels are indicated as « 1 percent; « 5 percent; 
* « 10 percent. 
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TABLE G.2 



IMPACTS ON YEAR-TWO EMPLOYMENT RATES AND EARNINGS FOR MEN, 

BY SITE 





Samnl e 
Si ze 


Experimental s 


Controls 


Impact 


p« 






Ever Etnploved. Months 13-24 






Concurrent 












At 1 an ta JOu Lurps 




93.3% 




-10.7 


0.464 


tLi/odn Jose 


77 


oo c 
88 . 3 


07 C 


1.0 


0.905 


Chicago Conmons 


42 


96.4 


88.6 


7.8 


0.483 


Connelley (Pittsburgh) 


.86 


89.3 


91.1 


-1.9 


0.812 


tOSt LM Oikl 1 Id LcriLcr 




81.1 


90.5 


-9.4 


0.336 


LuUb v^enverj 


DO 


75.6 


82.9 


-7.3 


0.416 


rnoeni a jqu ^urps 


Q J 


80. 7 


QQ ^ 
OO . J 


-7.5 


0.415 


SER/Corpus Christi 


143 


76.3 


7b. 7 


0.7 


0.911 


Sequential 1 in-house 












El Centro (Dallas) 


71 


92.9 


/9 .0 


13.8 


0.110 


LA JGD LOrpS 


oo 


71.3 


/D . 9 


-5.6 


0.468 


sequent laL / oroKei ea 












Al 1 entown iBUTi ai oj 


Do 


83.5 


00*0 


-2 5 

Cm • ^ 


0.793 


OCA fki^r\ 
dSA INYtJ 


DO 


89.6 


88.8 


0.8 


0.926 


tKtt inariTorcij 




107.1 


77.4 


29.7** 


0.022 


r^VQLUe Of r~5ZQZl5ZlC 








0.459 








Total Earninqs. 


Months 13-24 






Concurrent 














25 






-4,01^.55* 


0.054 


1 / oof] JOSc 


77 






553.74 


0.635 


Chicago ConiDons 


42 


6.894.20 


6,273.93 


620.27 


0.695 


Connelley (Pittsburgh) 


86 


2,667.97 


4,191.36 


-1,523.39 


0.168 


cod w hjy OKI 1 19 wciiwci 


56 


5,622.91 


8,340.70 


-2,717.79** 


0.049 


tuUo ^uenverj 




3,961.28 


4,915.83 


-954.55 


0.454 


rnoenix jod Lorps 


U J 


4,765.22 


4,847.08 


-81.86 


0.950 


SER/Corpus Christi 


143 


3,509.92 


3,421.01 


88.91 


0.918 


Sequential 1 in-house 












El Centro (Dallas) 


71 


4,714.69 


5,289.66 


-574.97 


0.639 


LA Job Corps 


88 


4,570.40 


7,537.41 


-2,967.01*** 


0.007 


Sequential /brokered 












All entown (Buffalo) 


58 


5,198.10 


4,480.79 


717.31 


0.598 


BSA (NYC) 


66 


8,154.38 


8,880.16 


-725.78 


0.565 


CREC (Hartford) 


31 


8,844.29 


6,040.01 


2,804.28 


0.127 


P-value of F-statistic 








0.141 





(continued) 
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TABLE G.2 (continued) 



SOURCE: MDRC calculations from JOBSTART enrollment form and survey data* 

NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not pArticipate. 

Random assignment did not always take ^ ace on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted 
means from two-way analysis of covariance procedures controlling for 19 kinds of difference 
in characteristics, other than site, before random assignment. The two categories used as 
factors were research assignment and site (see Ostle, 1975, p. 454). There may be slight 
discrepancies in reported sums and differences of these adjusted means because of rounding, 

*A two-tailed t-test was applied to each within-site impact. An F-test was 
applied to the interaction between site and experimental or control status. The p-value of 
the F-statistic is the probability that site impacts are different only because of random 
error* Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * 
= 10 percent. 
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TABLE G.3 

IMPACTS ON YEAR-TWO EMPLOYMENT RATES AND EARNINGS 
FOR WOMEN LIVING WITH THEIR OWN CHILD(REN) 
AT THE TIME OF RANDOM ASSIGNMENT, BY SITE 



Site or Statistic 


Sample 
Size 


Experimental 


Controls 


Impact 








Ever Employed. Months 13-24 






Concurrent 












Atlanta Job Corps 


18 


39.5% 


49.1V 


-9.6 


0.689 


CET/San Jose 


10 


86.1 


36.4 


49.7 


0. 125 


Chicago Commons 


18 


60.9 


46.8 


14.1 


0.553 


Connelley (Pittsburgh) 


63 


53.3 


31.6 


21.7* 


0.084 


East LA Skills Center 


12 


53.0 


4.4 


48.7 


0.138 


EGOS (Denver) 


67 


46.0 


37.8 


8.2 


0.500 


Phoenix Job Corps 


37 


46.3 


43.6 


2.7 


0.868 


SER/Corpus Christi 


53 


58.1 


69.7 


-11.6 


0.394 


Sequential / in-house 












El Centro (Dallas) 


51 


61.7 


49.4 


12.3 


0.393 


LA Job Corps 


66 


54.8 


44.9 


9.9 


0.422 


Sequential /brokered 












Allentown (Buffalo) 


46 


56.2 


62.6 


-6.4 


0.661 


BSA (NYC) 


19 


44.3 


58.5 


-14.2 


0.536 


CREC (Hartford) 


24 


40.9 


25.8 


15.1 


0.457 


P-value of F-statistic 








0.676 








Total Earnings. 


Months 13-24 






Concurrent 












Atlanta Job Corps 


18 


$1,863.38 


$1,452.36 


411.02 


0.813 


CET/San Jose 


10 


3,022.81 


2,686.77 


336.04 


0.886 


Chicago Commons 


18 


2,002.47 


2,074.73 


-72.26 


0.966 


Connelley (Pittsburgh) 


63 


755.92 


459.09 


296.83 


0.743 


East LA Skills Center 


12 


2,521.82 


305.39 


2,216.44 


0.349 


EGOS (Denver) 


67 


1,903.33 


1,973.14 


-69.82 


0.936 


Phoenix Job Corps 


37 


1,208.87 


1,969.39 


-760.51 


0.520 


SER/Corpus Christi 


53 


1,575.21 


2,324.46 


-749.25 


0.446 


Sequential lin-house 












El Centro (Dallas) 


51 


4,349.84 


1,959.85 


2,389.99** 


0.022 


LA Job Corps 


66 


2,450.62 


1,713.48 


737.13 


0.404 


Sequential 1 brokered 












Allentown (Buffalo) 


46 


2,162.68 


2,379.27 


-216.59 


0.837 


BSA (NYC) 


19 


1,992.69 


5,076.42 


-3,083.73* 


0.063 


CREC (Hartford) 


24 


1,894.11 


1,385.64 


508.47 


0.729 


P-value of F-statistic 








0.492 





(continued) 
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TABLE G.3 (continued) 



SOURCE: KDRC calculations from JOBSTART enrollment form and survey data. 

NOTES- Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data, including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For 
sme sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted 
means from two-way analysis of covariance procedures controlling for 19 kinds of difference 
in characteristics, other than site, before random assignment. The two categories used as 
factors were research assignment and site (see Ostle, 1975, p. 454). There may be slight 
discrepancies in reported sums and differences of these adjusted means because of rounding. 

^A two-tailed t-test was applied to each within-site impact. An F-test was 
applied to the interaction between site and experimental or control status. The p-value of 
the F-statistic is the probability that site impacts are different only because of random 
error. Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; 
= 10 percent. 
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TABLE G.4 



IMPArrS ON YEAR-TWO EMPLOYMENT RATES AND EARNINGS 
FOR WOMEN NOT LIVING WITH THEIR OWN CHILD (REN) 
AT THE TIME OF RANDOM ASSIGNMENT, BY SITE 



Site or Statistic 



Sample 
Size 



Experimental s Controls 



Impact 



Concurrent 

Atlanta Job Corps 18 

CET/San Jose 55 

Chicago Conmons 14 

Connelley (Pittsburgh) 35 

East LA Skills Center 32 

EGOS (Denver) 51 

Phoenix Job Corps 30 

SER/Corpus Christi 40 

Sequential /in-house 

El Centro (Dallas) 33 

LA Job Corps $4 

Sequen tial /brokered 

Allentown (Buffalo) 36 

BSA (NYC) 34 

CREC (Hartford) 32 

P-value of F-statistic 



Concurrent 

Atlanta Job Corps le 

CET/San Jose 55 

Chicago Conmons 14 

Connelley (Pittsburgh) 35 

East LA Skills Center 32 

EGOS (Denver) 51 

Phoenix Job Corps 30 

SER/Corpus Christi 40 

Sequential /in-house 

El Centro (Dallas) 33 

LA Job Corps 64 

Sequential /brokered 

Allentown (Buffalo) 36 

BSA (NYC) 34 

CREC (Hartford) 32 

P-value of F-statistic 



Ever Employed. Months u.?a. 



77.6V 

79.8 

88.4 

69.0 

67.8 

62.7 

58.7 

58.9 



69.8 
73.5 



72.9 
68.3 
66.4 



Total Earnings. Months 13-24 



70.9V 


6 7 


n 7 AO 
u • / Do 


71.2 


8.6 


0.457 


74.3 


14.2 


0.588 


39.9 


29.1* 


0.068 


50.1 


17.7 


0.294 


66.8 


-4.2 


0.748 


86.0 


-27.3 


0.124 


61.3 


-2.3 


0.875 


49.5 


20.2 


0.236 


64.4 


9.2 


0.434 


59.8 


13.2 


0.432 


69.1 


-0.8 


0.961 


95.3 


-28.9* 


0.086 



$2,212.27 
6,440.66 
4,829.92 
2,121.38 
4,504.55 
1,667.32 
1.764.39 
1,121.12 



2,792.78 
5,480.17 



2,971.29 
2,975.79 
3,795.73 



$4,800.24 
5,824.94 
1,498.89 
670.34 
2,091.86 
2,741.52 
3,590.86 
1,566.22 



1,838.33 
3,832.21 



1,622.63 
3,362.20 
7,219.70 



0.426 



-2,587.98 
615.72 
3,331.03 
1,451.03 
2,412.69 
■1,074.19 
■1,826.47 
-445.09 



954.46 
1,647.96 



1,348.66 
-386.42 
■3.423.97*" 

0.104 



0.203 
0.550 
0.153 
0.306 
0.109 
0.355 
0.247 
0.738 



0.531 
0.115 



0.368 
0.790 
0.023 



(continued) 
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TABLE G.4 (continued) 



SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all sample members for whom there 
were 24 months of follow-up survey data^ including those with values of zero for outcomes 
and those who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For 
some sample members, the month of random assignment may be less than a month, beginning with 
the date of random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted 
means from two-way analysis of covariance procedures controlling for 19 kinds of difference 
in characteristics, other than site, before random assignment. The two categories used as 
factors were research assignment and site (see Ostle, 1975, p. 454). There may be slight 
discrepancies in reported sums and differences of these adjusted means because of rounding. 

^A two-tailed t-test was applied to each within-site impact. An F-test was 
applied to the interaction between site and experimental or control status. The p-value of 
the F-statistic is the probability that site impacts are different only because of random 
error. Statistical significance levels are indicated as » 1 percent; ** = 5 percent; * 
= 10 percent. 
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